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CRITERIA FOR INDICATORS OF CHILD WELL-BEING 



Introduction 

Are the circumstances of children growing worse, or not? Are U.S. families falling apart, 

or not? 

How people answer, these questions will vary substantidly, depending upon where they 
live, their economic situation, their age, their personal values, their political party affiliation, and 
whether their focus is on trends in children's health, education, income, or family size. 

Yet, I could ask similar questions about the economy and get relatively consistent 
answers from different people. The economy, like families, is diverse and complex. Different 
regions enjoy quite different levels of prosperity, and different industries have quite varied 
earnings situations and prospects for the future. Yet we have a set of economic indicators that 
are quite widely accepted as markers of the country's well-being: the rate of inflation, the GNP, 
the unemployment rate, the Dow Jones average, and the poverty rate, for example. Our common 
vmderstanding of these indicators guides public policy, influences the directions of private 
organizations, and affects individual decision-making as well. We take economic indicators 
seriously, even when real concerns exist about their validity (Ruggles, 1990). Indeed, 
considerable resources are invested in maintaining and monitoring economic indicators. 
Information is updated regularly, and trends are reported monthly in some cases. 
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The status of economic indicators stands in stark contrast to indicators of child well-being 
where, despite earlier efforts (e.g., Watts and Hernandez. 1982; Select Committee on Children 
Youth and Families, 1989), we continue to lack not only a clear set of reliable, valid, up-to-date 
indicators but a consensus regarding what it is desirable to track. What set of measures can be 
developed that reflect a public consensus and that reliably and folly track the well-being of 
children and famiUes with children? This task is, ofcourse, the mission of this conference. To 
serve this mission, we need a concrete goal. Specifically, we need to think through what a 
system of indicators about children ought to look like. 

As an initial attempt at this task, I have developed a series of criteria for child indicators 
(see Figure One). I haven't tried to rank their importance, and I recognize that it may not be 
possible to maximize them all, particularly in the short run. They are put forward to provide a 
common point of departure. 




FIGURE ONE 

OUTERIA FOR INDICATORS OF CHILD WELL-BEING 

1 . Comprehensive Coverage. Indicators should assess well-being across a broad array of outcomes, 
behaviors, and processes. 

2. Children of all ages. Age-appropriate indicators are needed at every age from binh through adolescence 
and covering die transition into adulthood. 

3. Clear and comprehensible. Indicators should be easily and readily understood by die public. 

4. Positive outcomes. Indicators should assess positive as well as negative aspects of well-being. 

5. Depth, Breath and Duration. Indicators are needed that assess dispersion across given measures of well- 
being children's duration in a status, and cumulative risk foctors experienced by children. 

6. Common interpretation. Indicators should have the same meaning in varied population sub-groups. 

7. Consistency over time. Indicators should have the same meaning across time. 

8. Forward-looldng. bidicators should be collected now that anticipate the future and provide baseline data 
for subsequent trends. 

9. Rigorous methods. Coverage of the population or event being monitored should be complete or very 
high, and data collection procedures should be rigorous and consistent over time. 

10. Geographically detailed. Indicators should be developed not only at the national level, but also at the 
state and local level. 

1 1 . Cost efficient Aldiough investments in data about U.S. children have been insufficient, strategies to 
expand and improve die data system need to be thoughtful, well-planned, and economically-efficient 

1 2. Reflective of social goals. Some indicators should allow us to track progress in meeting nadonal, sUte and 
local goals for child well-being. 

1 3 . Adjusted for demographic trends. And, fmally, to aid with our interpretation of indicators, indicators, or 
a sub-set of indicators, should be developed that control or adjust for changes in the composition of the 
population over time which confound our ability to track well-being. Alternatively, indicators should be 
available for population sub-groups that are sufficiendy narrow to pennit conclusions within that sub- 
group. 
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Criteria for Social Indicators of Child Well-being 

The firet criterion for a set of child indicators is comprehensiveness. 

• Indicators should assess well-being across a broad array of outcomes, 
behaviors and processes. 

Indicators from only one domam provide an incomplete and potentially biased 

perspective on the development and well-being of famiUes and children. For example, while 

childhood mortality from communicable diseases has declined markedly, sexual activity and 

substance use pose new risks to adolescents. Similarly, while SAT scores have declined over 

time, improving only marginally for some sub-groups, conditions of housing have improved 

substantially for most famiUes. thus, a strong set of indicators needs to include measures from 

varied domains, including: 

population composition and geographic distribution; fertility; mortality; 

physical health and safety; access to health care; nutrition; 

mental health; access to treatment or counselling; 

educational attainment and cognitive achievement; school and non-school; 

cognitive stimulation and learning experiences; 

econonodc well-being; housing adequacy; assets; receipt of public transfers; 
behavior problems, such as delinquency and substance use; positive behaviors; 
attitudes; goals; 

family structure; contact with an absein parent (if any) and extended family 
members; child support; parent-child interaction and time use; chdldren in 
institutions and foster care; 

work force behavior of parents and children; child care for children; job-related 
benefits; and 

the child's school and neighborhood environment; 
The concepts covered within each domam can and sometimes need to be represented by 
numerous indicators. In addition, the same domain can be examined with a focus on either assets 
and strengths or on deficits and problems. In addition, indicators are needed for children of 
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varied ages. This leads to ova second criterion. 

• Age appropriate indicators are needed at every age from birth through 
adolescence and covering the transition into adulthood. 

Both the measures chosen and the data collection strategies employed need to vary for 
children of different ages. Infants, children in elementary school, and teenagers differ 
dramatically, and different indicators are needed to capture this variation. For example, 
accidents and safety in the home are more relevant for pre-schoolers, while motor vehicle 
accidents and substance abuse are more relevant for teenagers. 

The kmds of data required jfrom younger children must generally be supplied by parents, 
for example, the recency of medical care, accidents, and behavior problems. However, child 
assessments have been conducted in the homes of young children by especially trained survey 
interviewers among children age three and even younger in several instances. Teachers become 
an excellent source of data once children reach school age. Parents continue to be an important 
source of information about the family into the teen years, but increasingly it is essential to query 
youth themselves. The need to interview teens in most obvious for risk-taking behaviors such as 
substance use, sexual activity, and delmquency. However, e^'en for more neutral topics, such as 
time spent watching television, the older youth is likely to provide more accurate data than the 
parent. 

It is also important, if child mdicators are to be a part of pubiic discussion and policy 
debate, that they make sense to a lay audience, which represents the third criterion. 
• Indicators should be easily and readily understood by the public. 
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We cannot expect policy makers and the general public to^spend much time mastering 
complex and non-intuitive measures of well-being. Ideally, comprehension should be 
instantaneous. It should be possible for a person with an average education to more or less read a 
sentence and understand the trend. 

Of course, it is not possible to cover every topic with simple and readily accessible 
measures. Hence, it will often be necessary to have an additional set of relatively more complex 
measures that can be mastered with some moderate level of effort that is distinct from the set that 
is broadly disseminated in the media. However, even then, indicators should have face validity. 
We do not want to divert attention away from the meaning of the trends to explaining and 
justifying the indicator itself 

It is the case, of course, that many economic indicators lack face validity; but over time 
people have become used to measures such as "the poverty rate" and GNP and think they 
understand them. So we should not allow ourselves to be driven to suggesting only the most 
obvious and simple indicators above all else. We can also work to educate the public, the media 
and policy makers. 

The American press tends to highlight negative trends, and the public tends to show 
particular concern about negative trends. The focus on bad news and failures may account in 
part for the public's pessimism and the widespread belief that programs are ineffective. 
Therefore: 

• Indicators should assess both positive and negative aspects of well-being. 

The government itself, because it is organized to identify needs and to address and solve 
problems, tends to collect data on problems, such as crime, disease and death. So we hear about 
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trends in drug use, violence, and sexual activity; but we rarely hear about the commitment of 
youth to recycling, about the work youth do around the house or outside, about close parent-child 
relationships, or the time youth spend volunteering. We don't even hear much about neutral 
items, such as whether and how much allowance kids receive. Rarely, though, do parents 
content themselves with only the aspiration that their child not do drugs or just avoid being 
imprisoned. Most parents have positive aspirations that their child will be useful, well-adjusted 
and happy. 

To provide a more balanced perspective, new measures need to be developed and new 
data need to be collected. Possible constructs include community mvolvement, volunteering, 
religious practice and activities, exercise and sports, activities in school clubs, reading and 
cultural participation, family activities, and work done in the home. 

The lives of children are complex and multifaced. Our indicators should reflect the 

complexities of children's lives. We know that long-term poverty is more problematic than 

short-term spells of poverty. We know that cumulative risks undermine children's development 

more than a single risk. And we know that average values do not reflect the diversity 

experienced by children from varied backgrounds. Therefore, our fifth criterion is: 

• Indicators are needed that assess dispersion across given measures of well- 
being, the duration that children spend in a given status, and which assess 
cumulative risk factors experienced by children. 

Measures of duration can be developed to assess how long children live in varied 

statuses, e.g., how long they have lived in poverty, the years children spend in a single parent 

family, the months or years that children have a chronic condition, or the time children live witl 

both biological parents. For this task, longitudinal data are generally required. It is possible to 
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develop longitudinal indicators from retrospective data, but the quality of the data diminishes as 
the interval lengthens. In fact, annual or biennial data produce higher quality data than that based 
on data collected every five or six years (Moore, and Glei, 1993). 

Several studies have demonstrated that children experiencing more than one risk factor 
face very elevated probabilities of poor outcomes (Samerofif, Seifer, Barocas, Zax and 
Greenspan, 1987; Moore, Nord and Peterson, 1989). However, few indicators assessing the 
proportion of children facing multiple risks are currently available. The data required to estimate 
multiple risks can come from demographic variables, such as income, parent education, and 
family structure; but they might also be based on measures of family processes, such as 
monitoring of the child's activities, parent/child joint activities, and parent involvement in school 
events. A very strong measure of risk could be developed on the basis of duration in multiple 
risk statuses, for example, the number of years a child lived in a single parent family in poverty 
with four or more siblings. 

Having developed these (and other) complex indicators of child well-being and risk, it is 
usefril to go beyond just universally reporting the proportion at the bottom or the average status 
of children. It is valuable to emphasize heterogeneity, to illustrate the varied experiences that 
contemporary childi en have living in advantageous or disadvantaged circiunstances. It is as 
helpful to know, for example, trends in the proportion of children in environments with no or 
only one major risk factor, as it is to know trends in the proportions of children growing up 
encumbered by multiple risks. 
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Whether a particular indicator measures well-being over the short-term or the long-term, 
whether it assesses single or multiple risks, the meaning of the indicator should be clear to all. 
This is the sixth criterion. 

• Indicators should have the same meaning in varied societal groups. 

Infant mortality is a tragedy that can be measured and tracked with considerable and 
comparable acciuacy across racial, educational, and income groups. Whether the same can be 
said for other topics such as attitudes about gender roles, discipline patterns, aspirations, and 
religiosity is an empirical question. 

Little methodological research has been conducted to examine whether these kinds of 
indicators hold the same meaning across different groups. This is unfortunate because these are 
iareas where American society seems to be undergoing substantial change, and these changes may 
have important implications for family life and children's development. In the short run, reliance 
on behavioral indicators represents the most cautious approach. For example, rather than 
assessing religious belief, a nebulous concept, it may be necessary to be more tangible and assess 
actual attendance at religious observances or services. Measures of behavior in general will be 
less susceptible to differential meaning across societal groups than measures of feelings and 
attitudes. However, methodological research is needed to develop solid measures of attitudes, 
values, and goals, as well as behaviors. 

The seventh criterion is particularly important if our goal is to develop a set of indicators 
that can be used into the next century. 

• Indicators should have the same meaning over time. 
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Where social indicators are concerned, it is a real challenge to develop indicators that 
hold their meaning across the decades. Such a fundamental measure as the proportion married, 
for example, has changed as formal marriage has been postponed but often been replaced by 
cohabitation. This change over time affects not only indicators of marriage and family structure 
but measures of fertility, such as the non-marital birth rate. For some measures, such as family 
structure, researchers can go back and re-analyze data to provide a consistent measure over time, 
to the case of remarriage, for example, household records can be searched to identify cohabitmg 
partners for a measure of family structure. However, similar re-analyses are not possible with 
vital statistics data. 

As social conditions change, it is often tempting to abandon existing measures in favor of 
new mdicators that more accurately describe contemporary society, to many cases, new 
measures are essential. The cost is that we cannot track trends if we change measures. A 
compromise in strategy is to collect information that permits tracking of both the origmal and the 
new mdicator. An example can be provided from work on adolescent sexual activity. 

Available indicators do not distinguish between voluntary and non-voluntary initiation of 
sex. However, research indicates that a substantial proportion of the first sexual experiences of 
girls are non-voluntary (Moore et al., 1989). Whether first sex is voluntary or coerced has, of 
course, substantial implications for services and policy, and it seems imperative that surveys 
collect information that makes a distmction between voluntary and non-voluntary sexual 
initiation. To change the defmition of sexual debut, however, destroys historical continuity and 
makes it impossible to assess trends m adolescent pregnancy among those at risk of pregnancy. 
Thus, collecting data that permit analysts to construct both the traditional and new indicators 
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seems imperative. If it is not feasible to collect such extensive data on a regular basis, it should 
be possible to collect the data needed to maintain the historical measure on an intermittent basis. 
This discussion leads to the eighth criterion. 

• Indicators should be collected now that anticipate the future and provide 
baseline data for subsequent trends. 

Society is becoming more technological and international. The availability of complex 
technology in the home, such as high-powered personal computers, laser printers, and 
educational games, is now a critical component of parenting that may give middle class children 
a tremendous lead over low income children. Similarly, travel to foreign countries and mastery 
of foreign languages and customs are becoming more common for families and for students in 
some school programs. Also, post-secondary education outside of college settings has become 
widespread, and such training can have important implications for employment and income. 

We all "know" that these kinds of changes are occurring. Unfortunately, over-time data 
with rigorous and consistent definitions and methods are not available to dociiment such trends. 
We need to think about other changes that are occurring that seem likely to continue and to begin 
now to develop measures that can provide a baseline for the future. 

The ninth criterion requires little elaboration. 

• Coverage of the population or event being monitored should be complete or 
very high; data collection procedures should be rigorous and should not vary 
over time. 

Response rates in American surveys have declined over the years, and this decline poses 
an issue for the continuity of some data series. Recently, the OfTice of Management and Budget 
has opposed payment of incentives for respondents, one strategy which might increase response 
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rates. Consideration of the response rate in a survey should affect whether that survey is viewed 

as a source of indicator data. 

Changes in data collection methods can also affect reporting. For example, reporting of 
substance use by teens is higher when they are able to enter data directly into a laptop computer. 
This methodological improvement thus appears to enhance data quality for highly sensitive 
topics. At the same time, however, changing over to this new method may undermine trend data. 
It is essential, of course, to improve methods and procedures for collecting data. But we must 
develop methods for calibrating the new measures with old measures, or intermittentiy repeat the 
old measures, if we are to mamtain the integrity of a social indicators series. 

• Indicators should be developed not only at the national level, but also at the 
state and local level. 

In order to assess the effect of policies and services, of changes m services and policies 
and of social, ecological, and economic variations at the state and local level, indicators are 
needed that track the status and well-being of families and children at levels of geographic and 
political aggregation below the national level, hi particular, measures are needed at the state 
level. With education organized at the local level, even state-level data are inadequate. At 
present, the vital statistics system and the decennial Census provide the primary sources of local 
area data. As state agencies computerize and upgrade their record-keeping, state child support, 
welfare, health and education data systems may generate much useful information at the state and 
sub-state level that can be used to track family and child well-being. 

Investments in data regarding children and famUies have been modest, relative to public 
and private expenditures on otiier groups such as tiie elderly, on current consumption, or on otiier 
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social problems such as the savings and loan crisis. Hence it is assumed that some spending 
mcreases are appropriate, especially in view of current concerns regarding the well-bemg of U.S. 
children. Yet, with a substantial Federal deficit, high taxpayer resistance to new taxes, ways to 
hold down the cost of obtaining and processing indicators data must be considered thoughtfully. 

• While cost should not be the driving force behind decision-making, planners 
should consider cost as design decisions are made. 

Costs can be minimized by employing data for the purpose of social indicators that were 
originally collected for other purposes, such as the AFDC Quality Control Survey, where data are 
collected primarily to monitor welfare fraud and mistakes. This can also be done by 
piggybacking a child module onto other surveys being conducted for other purposes, such as the 
child module now being developed for the Survey of Income and Program Participation, 
conducted to assess income and employment and the receipt of income supports and services 
among individuals and families. Similarly, the child health supplements added periodically to 
the National Health Interview Survey, provide information on children's physical health not 
elsewhere available. In addition, data collected for analytic purposes, such as the National 
Longitudinal Survey of Youth, a labor force survey, and the National Educational Longitudinal 
Survey conducted by the Department of Education, can also be analyzed to provide cross- 
sectional indicator data. 

Indicators should also reflect societal goals and support the efforts of policy makers to 
assess progress toward meeting national goals, such as school readiness, economic self- 
sufficiency and comprehensive health care. The twelfth criteria reflects this need. 

• Indicators should help track progress in meeting national goak for child 
well-being. 
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Because many programs and policies are initiated and managed at the state and local 
level, it is also important to measure and track indicators of child well-being at these levels. 
Currently in the U.S., sustained attention is focussed on health care and welfare receipt. Efforts 
to improve indicators of child well-being should inform these policy efforts. Similarly, concern 
with violence, drug use, scholastic achievement, immunization, and adolescent pregnancy argue 
for up-to-date information on these topics. 

Since indicators cannot be developed for every aspect of children's behavior, choices 
must be made. Indicators should focus on issues of general Or substantial importance to the 
public, program providers and policy makers. Alternatively, indicators, or at least a subset 
of indicators, should be weighted by the importance or salience of an item to the public or by the 
proportion of the public affected by a trend. 

Victory over rare congenital diseases represents an important event for the families 
affected, but as social indicators it is more useful to assess trends in more widespread conditions, 
such as low birth weight, sexually transmitted diseases, accidents, etc., that affect large numbers 
of individuals. 

Finally, to aid with our interpretation of indicators: 

• New indicators, or a sub-set of indicators, should be developed that control 
or adjust for changes in the composition of the population over time which 
confound our ability to track well-being. Alternatively, indicators should be 
available for population sub-groups that are sufficiently narrow to permit 
tracking within that sub-group. 

The composition of the population affects our interpretation of trend data. For example, 
the United States has experienced a substantial influx of Hispanic persons into the country and 
has simultaneously seen the white teen birth rate rise. Since 90 percent of all Hispanics are 
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classified as white by the vital statistics system, separating non-Hispanic whites from Hispanics 
shows that much of the increase in the white teen birth rate is due to increased numbers of 
Hispanic teens. Only if the data are analyzed by race and ethnicity sunultaneously is this subtle 
but important difference apparent. 

Sinularly, trends in SAT (Scholastic Aptitude Test) scores have been a source of 
substantial concern in the United States, as college-bound high school students, with some 
exceptions, have tended to score lower tlan students did several decades ago. When the data are 
analyzed controlling for changes in the composition of the student body taking the SAT, some of 
this negative trend can be explained. It is important to know that some of the change can be 
explained by changes in demographic composition, but it is also unportant to know that not all of 
the change is explained by changes in composition. 

Hence, when trends are evident that are particularly puzzling or troubling, re-analyzing 
the data within narrow population groups or using multivariate methods can be very helpful in 
explaming the meaning of these trends. 

Summary 

In the United States, efforts to develop, tabulate and disseminate social indicators 
regarding children and families have moved forward erratically. There appears to be a renewed 
interest at the present time in assessing the circumstances under which children are growing up. 
Traditional indicators are receiving new attention, and an effort is being made to develop a richer 
set of indicators. Developing a complete and rigorous set of measures appropriate to the next 
century is a challenging task; but if you believe, as I do, that the well-being of children is as 
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crucial to the nation as the well-being of the economy, then it is appropriate to direct the needed 
talent, energy, and resources to the task. 
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Purpose and Background 

In this paper we outline a set of (rebuttable) propositions about how indicators of child well-being 
might be created by social scientists and used by advocates who play by different rules than social scientists. In 
particular, we are concerned with indicators for troubled children who find themselves in the social service 
system because of delinquency, abuse or neglect, or educational failure. We refer to vhese indicators as 
"psycho-social* indicators of child well-being. 

We have four aims. First, we try to clarify terms and discuss the evolving history of indicators. 
Second, we look at the multiple uses of indicators. Third, we explore and critique exan^iles of psydio-social 
indicators now used, noting strengths and weaknesses. Finally, we make recommendations concerning 
iiq>rovement. 

Background 

The desire to gaJier informadon to report the well-being of children and families has a long history. 
Foundations, advocacy f roups, states, and the Federal government are engaged in numerous activities to 
develop and refine social: stsiistics for a host of reasons. Some or all of these activities use the word 
"indicators* synonymously with social statistics or as subset of them. The U.S. Department of Health and 
Human Services, in conjunction with the Institute for Research on Poverty, CMd Trends, The Annie E. Casey 
Foundation, and the National Institute of Child Health and Human Development have organized a set of 
activities to in^rove the conceptual foundation of indicators and their rqwrting. These activities include 
conceptual work on indicators and planning for in^roved indicators. 

In November, 1994, a conference was held to discuss conceptual work on developing indicators. 
Authors of conference pa^ were asked to address (1) the state of indicators, (2) the prospects for in^roved 
indicators by the end of the decade, and (3) the resources and methods required to obtain better indicators. The 
subject areas assigned to the authors-such as health, education, and income security-touched only tangentially 
on issues associated with children caught up in the "child welfare" and related systems. This system includes 
children experiencing abuse or neglect, those who exhibit anti-social behaviors, and those who require social 
services or out-of-home placement. Though the number of children involved in these systems may be small, 
their conditions represent the antithesis of "well-being." 

We acknowledge the many thoughtful explorations of indicators by those who have preceded us, and 
we are humble about what new we bring to the discussion. We hope that our presentation provides a somewhat 
new perspective that enhances the debate over the utility of indicators of dtild well-being and leads to the 
iiiq>rovement of these indicators. 



Wliat are indicators and whv have them?' 

Definition of terms 

We feel comforuble with the following defmition of "indicators": "quantitative </eJcn>ftonj of social 
conditions that are intended to inform public opinion and national policy making" (Duncan, emphasis added). 
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' The discussion that follows is intended to relate to indicators in general, not just psycho-social \dicators. 

^ The Government Performance Review Act (GPRA) discusses indicators and associates them with several 
types of dau: an outcome is an assessment of the rcsulu of a program compared to iu intended purpose. An 
output is a measure of an activity or effort that can be expressed in quantitative or qualitative terms. An irt^i 
is the comparison of program effects/consequence/outcomcs with estimates of outcomes that would have 
occurred in absence of the program. We have encountered a great deal of confusion concerning whether GPRA 
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"Describing" is a more limited vision, pediaps more pragmatic and politically feasible, than measuring or 
evaluating program effectiveness. To us it idso means painting a fair and accurate picture of magnitude, 
composition, and trends. We believe this is intentionally a limited enterprise, free of advocacy objectives. 

"Conditions" also have an inqraitant connotation to us. John W. Kingdon in Agendas. A lternatives. 
and Public Policies makes a useful distinction between "conditions" and "problems." We put up with all kinds 
of conditions, including bad weather, poverty, and illness. Conditions become public problems only when 
govetnments-actively or in reaction to public demand-decide to do something about them: that is, when they 
become part of the government agenda (Kingdon, 1984). 

"Inform" can have several interpretations. We also use a limited version of it here: one more 
associated with policy analysis than policy advocacy. In our view, informing means making a balanced 
presentation of the dau. Policy analysis considers several variables and perq)ectives. It presents plausible 
interpretations. It strives to meet a test of fairness. The policy analyst attenq>te to be both prosecution and 
defense: making sure that all relevant perspectives are presented to the court. Policy advocacy, on the other 
hand, tries to influence the jury of public opinion. It makes the best case for its client: "influence" is 
substimted for inform. 

Indicators, generally, are a selected set of dau that are intended to bring insights to various audiences 
about our country's social problems and achievements. In our opinion, indicators differ from "social statistics." 
Social Statistics describe, while indicaton purposefiilly inform. The Statistical Abstract is a collection of social 
statistics. A small subset of these social statistics could be called indicators, but a set of indicators may well 
contam statistics not found in the Statistical Abstract . They are not measures of programmatic effort or inpute. 
In addition, they are not measures of programmatic outcome. As we discuss below, indicators cannot determme 
the impact programs have made on the lives of children, except in the broadest sense of the word "indicator." 

Indicators are used by different actors for different purposes and should be judged according to their 
purpose. While indicators most often are constructed and collected by social scientists or program 
administrators, they are used by a host of types of people, such as analysts, advocates, government workers, 
other experts, politicians, the media, and the general public. Their many uses may conflict. Different standards 
of fairness or effectiveness apply depending on use or purpose. Social scientists have a stake in assuring that 
these tools are reliably constructed and not misused. Some purveyors of indicators-advocates-hope to spur 
public action. The roles of the scientist and the advocate may conflict. We apply different standards to 
indicators as we define them than we would apply to advocacy or to program evaluation-uses others may 
associate with indicators. We discuss these different standards below. 

A brief histO Tv of social indicators 

Interest in social indicators has waxed and waned over the last half-centuiy. One of the most vigorous 
periods was the late 1960's and early 1970's during a time of great faith in social science. Social scientists in 
the social indicator movemem hoped to create a social index equivalent to the economic indices such as the 
GNP. They sought a common metric equivalent to dollan in economic analysis. The current Index of Social 
Health produced by Miringoff and his colleagues at Fordham University is an exanq>le of this vision (Miringoff, 
1993). 

A review of the early literature illuminates a grandiose vision of indicators and the prospects of social 

science: 

Social Indicators, the tools, ait needed to fmd the pathways through the maze of society's 
interconnections. They delineate social states, defme social problems, and trace social 



is part of or substantially different from the general interest in promoting the general use of indicators. 
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trends, which by social engineering, may hopefully be guided toward social goals, formulated 
by social planning (Stuart Rice, cited in Otis Dudley Duncan, 1974). 

Though the fervor for social indicators has been muted over the past 25 years, the 1990's find a host of new 
aqiplications of indicators. We have some trepidation about discussing these projecu, because they do not 
comport exactly with our proposed use of indicators. Yet, we feel that understanding the breadth of potential 
uses provides guidance to the construction of indicators and the caveats that must acconq>any any such 
enterprise. Below we outline a few of these efforts. 

One current use of indicators is to assist planning initiatives, to target resources, to evaluate the success 
of programs and initiatives, to hold government leaders and agencies acco u n t able, to inform the public, and to 
generate public support. The Government Performance and Reporting Act (GPRA) and Oregon's Benchmarks 
initiative ace exanq)les of this type of effort. The recent GPRA legislation mandates expanded use of 
performance measurement in the federal government. The act tries to promote strategic planning and in^roved 
resource allocation. It also requires agencies to develop outcome measures for most national programs. At the 
same time, the Vice President's National Performance Review seeks to "reinvent" government by incorporating 
the use of outcome indicators into agency practice, leading agencies to become more efficient and customer- 
oriented. 

Various states also have mounted efforts to develop measures to gauge people's lives and to set 
program and budget priorities. A good exan^le of these is the Oregon Benchmarks initiative, which involves 
citizens in setting goals, tracking indicators, and assessing progress. This use of indicators is now imbedded in 
agency planning and decision making in Oregon. This is both an indicator project and an exercise in 
participatory democracy. The public is an integral part of the process. 

Another type of project comes from outside government. For the last several years, the Annie E. 
Casey Foundation of Baltimore has sponsored the KIDS COUNT National Data Book. The introduction to 1994 
Data Book states its purpose: 

KIDS COUNT is a national and state-by-sute effort to track the status of children in the 
United States. By providing policy makers and citizens with benchmarks of child well-being, 
KIDS COUNT seeks to enrich local, state, and national discussions concerning w^s to secure 
better futures of all chUdren (KIDS COUNT National Dau Book, 1994). 

The foundation also funds projects in each state that report indicators of diild well-being at the county and 
community level. 

Our assessment of KIDS COUNT activities is that they are a mixture of informing and advocating. 
For exan^le, KIDS COUNT presents a number of social statistics, such as the percent of all births that are to 
single teens and the infant mortality rate. However, the book also ranks states from best to worst on all of the 
indicators. This seems to be advocacy-trying to influence the public debate. 

A final example is the Index of Social Health which tracks 16 measures of social progress and annually 
reporting results (Miringoff 1993). For the last ten years Miringoff and his associates have gathered dau on 
these 16 elements and combined them into a single number index-almost like the Dow Jones Industrial 
Average. We have serious reservations about this single number index since it requires weighting the various 
indicators in relative importance to each other. Such weighting requires value judgements we are unprepared to 
make. 

The role of indicators in oolicvmaking 

We believe that efforte to construct a set of indicators must consider how social science dau may 
infiuence government policy making and agenda setting. Advocates try to convince government policymakers 
that problems exist, that they have potential solutions, and that resources should be allocated to support these 
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solutions. Cieating indicators and selling their in^rtance is critical to moving an issue from merely a 
condition to the status of a problem (Kingdon, 1984). The process by which conditions get classified as 
problems is tmpredictable, however. Windows of opporumity for redefinition open for indeterminate reasons 
and remain open for uncertain periods. 

An advocate's selection and interpretation can be subjective and time varying. A condition becomes a 
problem when some incident, anecdote, or indicator deviates from an ideological (or normative) goal, or when it 
differs from what seems possible. For example, people may perceive a problem when large differences among 
denK)grq)hic groups occur, as with the large differences in infant mortality, out-of-wedlock childbearing, and 
marriage rates among the races. 

The tmpredictable nature of the policy making process creates a dilenuna for those of us trying to 
develop a set of indicators. Are indicators timeless statistics, regardless of policy priorities? Should indicators 
provide an information inventory that sits on a shelf to be used when needml? Or, should they be selected and 
tailored to the advocacy objective at hand? 

Advocacy versus social science 

Using indicators for advocacy, in our opinion, often brings conflicts with social science. One point of 
conflict is the transparency of indicators. In seeking to call attention to a problem, advocates prefer to use 
indicators that are easily understood by lay-people. Social scientists, in contrast, generally feel uncomfortable 
with indicators which promise transparency and simplicity. Th^ often want to develop more opaque indicators 
that control for one or several other factors and that are consistent with the tenants of social science. Such 
indicators may be more difficult for lay-people to understand. 

Another point of contrast occurs when advocates suggest sinq)le connections between programmatic 
inputs and conditions measured by indicators. Social scientists, in contrast, suspect "evaluations" that do not 
have iq)propriate controls or comparison counterfactuals. There may be some indicators that are built from 
objective data on child well-being and yet serve as advocacy tools to change the priorities of the agenda-setters. 
However, it is also possible these purposes are inherently incompatible. 

To illustrate the potential tensions, let us look at the example mentioned earlier: ranking states, from 
first to fiftieth, on the basis of child-bearing rates for teenage girls. It seems to us that the primary reason for 
ranking states based on these rates is to encourage action in the states with the worst rankings-to motivate 
ofHcials to respond to teenage child-bearing. Transformmg rates into st£te-by-state rankings appems to be done 
for advocacy rather than social science purposes. It is done to motivate rather than to inform. 

While raw rates of births to single teens may be more transparent, social scientists know that these 
rates are strongly influenced by the economic and social conditions and the racial conqwsitions of the population 
in the areas being con^>ared. Presenting rates controlled for race-ethnicity or income may be a more balanced, 
but opaque, ^roach. The more statistical transformation that an indicator goes through, the more opaque, but 
less misleading, it may become (Garbarino, 1991). 

State rankings may give a false sense of complacency to high ranking states and inordinate blame to 
low ranking states. A New England state, for exanq)le, might look quite high. Further investigation behind the 
raw numbers, however, might show that the particular sute is not doing nearly as well as would be expected 
given the composition of its citizens. Before policy analysts can determine whether resources should be 
allocated to the lowest raw-score state or lowest controlled-score state, they must know the answers to several 
other questions so that thry can maximize the retvjn on their resource investment. 

Indicators and evaluation 

It is hard to fault efforts to hold government officials accounuble for their performance. Yet, only in 
special circumstances can indicators be used to evaluate the intact of public policies. In addition, the more 
transparent an indicator is, the less likely it is to be directly connected to government action or inaction. 
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Indicators seldom pass two general evaluation requirements: (1) understanding how program 
interventions relate to results and (2) having a valid counterfactual. First, evaluating program effectiveness 
requires at least a rudimentary understanding of the causal linkages among such elements as recipient 
characteristics, program participation, program activities, and program outcomes. Generally, given the current 
state of social science, we have a "causal ignorance* of these inter-connections (DHEW, 1973). In conq)lex 
systems like local child welfare systems or state employment and training programs, sin^>le indicators cannot 
provide sufficient intelligence about the causal links or counterf actuals. 

Second, the lack of a valid con^arison presents a serious barrier to the use of indicators to evaluate the 
success of programs. The essence of the scientific method is the concept of a counterfactual. Evaltiations ask: 
how would the treatment subject (or the program recipient) have responded in the absence of the program 
intervention? Of course, we can never know that without a doubt. We cannot turn back the clock and see the 
client's life again without the program intervention. Sometimes, social scientists create clinical trials with 
randomly selected groups, one receiving the program and the other receiving "standard care.* Often, 
circumstances prevent us from conduaing a classical experimental trial, so we use conq>arison groups or other 
methods to create counterfactuals. Indicators should have some counterfactual as well. Unless an indicator is 
constructed with a valid conq;)arison, it cannot be appropriately used to evaluate the success of program policies, 
in our judgement. 

Given our definition of indicators, evaluation results must pass a higher scientific test of 
sqipropriateness than indicators. (We discuss in the next section the tests that we believe indicators must pass.) 
Indicators are the products of statistical reporting systems, surveys, or administrative records. They are not the 
products of in-depth, multi-variate studies of programs and recipients. As such, indicator trends may pronq>t 
studies of programs and policies, but the trends can never alone validate the success or failure of a policy or a 
program. As a consequence, we see only loose linkages between efforts to develop and use child well-being 
indicators and efforts to develop outcome-oriented measures of government performance, such as the National 
Performance Review and GPRA. 

A case might be made that indicators may be used to hold government accountable in a general sense- 
that "things* are getting better or worse. It is possible to cite examples where the use of an indicator for this 
purpose makes some sense. For example, the poverty rate may have been a valid measure of the success of the 
*War on Poverty* in the 1960s. But-like judging the performance of the President on the general state of the 
economy-such judgements may be erroneous and should be made cautiously. The program (like the President) 
may have very little influence over the measure (like economic conditions). 

While indicators probably cannot prove the success of a new program, they may be able to give 
warning signals that things are not working as planned. Like economic leading indicators, certain measures of 
well-being might be able to give warning signs. Such signs should then precipitate in-depth reviews. 



Criteria for good indicators 

We now turn to a discussion of the sqipropriateness of indicators used to measure the psycho-social 
development of children. First, we present specific propositions about psycho-social indicators now being used. 
We believe, however, that some of these propositions can be generalized to i^ply to other indicators. Second, 
we present some general propositions. We hope that these propositions can giiide efforts to develop indicators 
that tell us how well parents and communities are nurturing children and assuring satisfactory transitions to 
adulthood. In Appendix A we have compiled of set of standards of reliability, validity, and appropriateness thai 
might apply to indicators generally. 
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Propositions about Psvcho-Social Indicators 

In this section, we present some propositions specifically about psycho-social indicators and explore a 
few indicators now in use, con^aring them against our propositions. By psycho-social indicators we mean 
indicators for troubled children who fmd themselves in the social service system because of delinquency, abuse 
or neglect, or educational failure. 

The specific propositions discussed here suggest ideals that few indicators can conqiletely satisfy. The 
propositions overl;q> one another, and shortcomings in indicators may be bundled together. If an indicator fails 
one of the propositions, it may also fail others, though for slightly different reasons. Violations of one or more 
of the propositions imply that the presenter should warn the reader of the potential problems of the indicators. 
The reader should then consider lutw the violation may affect interpretation. An indicator that cannot fidfill one 
or more proposition may still have value, however. We believe it should not be automatically discarded. 
Rather, authors Should note the caveats. 

For each proposition, we cite some indicators now used to measure children's psycho-social well-being, 
and we discuss the pros and cons of these indicators. Table 1 lists the indicators we discuss. This list is meant 
to be illustrative of psycho-social indicators now used. It is not comprehensive, and it is not our proposal for a 
set of indicators. 

We draw most of the indicators in Table 1 from KIDS COUNT reports. The source of the indicator is 
listed next to the indicator. We selected examples from KIDS COUNT not to fmd fault with the projca, but 
because it represents one of the most conq)rehensive child indicators efforts in existence. 

The indicators we discuss do not include measures of direct psycho-social well-being of children 
through the administration of depression scales or other psychometric efforts. Though we ^plaud the efforts of 
those who are using survey techniques to gather such measures for a large number of children, we do not 
believe these measures yet provide indicators of child well-being for a broad population. If that becomes 
possible, such indicators may replace some of those we critique below. 

Direct measurement 

First, an indicator should directly measure a factor in a child's life, originating from someone close to 
the child and having little mediation from the service system. Measures of child well-being should accurately 
reflect the true condition of the child, not features of the human service delivery system. Two issues underlie 
this proposition. One is the source of the report: the child, a parent, a teacher, or someone in less fr?equent 
contact with the child. A second is mediation by a service system. 

Source of the report . Some measures rely on a child's self-report of his or her condition. For 
exanq)le, the youth unemployment rate is determined through direct surveys of young people.' More often, we 
rely on an adult proxy to represent the well-being of the child. Sometimes a parent is the proxy respondent; at 
other times a professional assessing the condition of the child is the child's proxy. 

A range of professionals assess child well-being. Teachers assess and report on children's leammg 
ability or educational progress. Police investigate victims' reports of criminal activities of youth. Child welfare 
professionals investigate and report on abuse and neglect. Physicians assess children's health. Psychologists 
assess children's emotional well-being and mental capacity. Shelters record information on the runaways they 
see. 



' In national surveys we seldom interview children. Fear of youths truthfulness and maturity or parental 
resistance are serious concerns. Child Trends' Survey of Children is a rare exception. 
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It is possible to think of a continuum of sources for understanding child well-being. At one end, 
reports come from the child's own perceptions of his or her condition. At the other end, professionals 
embedded within a service system assess and report on the child's condition. 



< > 

Child's own Parent's Indep. Prof. Service system 

report Report Assessment Asses3ment 

There is no correct place on this continuum. Rather, for different domains of children's lives, dififerent 
trade-offs exist at each point. CMdren are obviously closest to their own situation. However, th^ may not 
fully understand their condition, or they may not be able to communicate it. On the other side of the 
continuum, reporters may have professional expertise at identifying conditions, but they are removed from the 
child and more likely to be influenced by the service system context. Furthermore, th^ may only see a self- 
selected portion of the problem. For each indicator it is in^rtant to consider the tradeoffs at eadi point on the 
continuum. 

Mediation bv the service system . If indicators are to relate to child well-being, they should measure 
the child's condition rather than whether the service system adequately responds to the child or has sufficient 
resources to identify and treat a problem. With any proxy measure fh>m reporters within the system, the 
service system mediates the measure of a child's well-being. 

Many indicators of psycho-social well-being do not directly measure the condition of children. Rather, 
they measure the response of the service system to the condition of children. For example, the rate of 
confirmed child abuse and neglect measures, in part, whether the child welfare system has enough resources to 
accept and investigate complaints of abuse and neglect. Similarly, the juvenile arrest measures law enforcement 
effort and technique as well as criminal activity by juveniles. 

Mediation by the service system can occur at several points, as indicated in the flow outlined below. 
Community values may affect whether a problem is recognized by a potential reporter. The service system may 
determine whether it is then reported, whether it is confirmed, and whether it results in service. 

Problem Problem Report Service 
Recognized? — > Reported? — > Confirmed? > Response? 



There are at least four ways the service system mediates an indicator. First, time may constrain a 
reporter's action. Second, the supply of a service may constrain a reporter's action. Third, changing 
community values outside the service system {e.g., a change in the acceptability of spanking as proper 
punishment) may influence a reporter's action. Fourth, various factors may affect whether the reporter senses a 
problem and believes it to be worth the consequences if reported. In sum, how a service system defines the 
problems it addresses, depends in part upon what the systems desires to do and is able to do. 

Time constraints . In many service systems, actions of professionals are severely constrained by 
funding and time. Assessments of child well-being may be influenced by these constraints. The true condition 
of children may not be discovered because professionals are too busy to conduct the necessary measurement or 
because th^ lack incentives to find the time to look for these problems. 

Table 1 includes several examples of indicators that face this difficulty. For exan^le, the rate of 
special education receipt may not be a true reflection of the rate of children with special needs. In school 
systems where teachers are severely overburdened, many children who may benefit from these services may not 
receive them because they are never properly tested. Teachers do not have the time to identify such students. 
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Another exanqile comes ftom the juvenile justice arena. Police may have incentives not to book 
children for some offenses. It may be easier, and faster, to release young offenders back to their parents 
without going through formal proceedings. Therefore, the rate of anest of children may not reflect the true 
number of children committing offenses. (This is a variation of the fourth point discussed below.) This 
distortion is likely to be less severe with more violent offense, where police may feel they must go through 
formal arrest proceedings. 

Service sup ply y^nyt^ip^y Many indicators are influenced by the supply of, and the demand for, a 
service. The rate of service reported may be a poor proxy for the underlying condition of children. A great 
supply of the service, in the face of low demand, may lead to children receiving the service who do not face the 
underlying condition. A constrained supply of the service, in the face of high demand, may lead to far fewer 
childrni receiving the service than those who face the underlying condition. This is a problem of the system 's 
resources, not of the workers' time and personal resources. 

The rate of substitute care placement, for exan^le, is an indicator where supply and demand may 
distort the measure of the underlying condition. Children should be placed into substitute care because workers 
believe that they will not be safe in their own homes. The rate of substimte care is easy to measure, but it is an 
indirect proxy for the rate of children who do not have parents who can properly care for them. It is, in 
essence, a proxy for the rate of severely inq)roper parenting. A household survey of parental behavior-a much 
more conq;>licated and expensive measure-would possibly provide a very different measure of iiq)roper 
parenting. 

In jurisdictions where the supply of foster homes is less than the demand for them, the rate of substitute 
care will uitder-report the rate of severely inq)roper parenting. In areas where foster homes are relatively 
plentiful, the rate of substimte care may over-report the rate of severely inq)roper parenting as child protection 
workers place children who may not truly need to be placed. This situation is now con^ilicated by die use of 
'kinship* foster homes, where children are taken from their parents and placed with close relatives. The 
increased use of 'kinship* homes has greatly increased the supply of sut»timte care and may distort time trends 
in this indicator. 

Changing c nmmiinitv values Professionals within the service system are influenced by changing 
community standards over time. Differences in an indicator over time, or across jurisdictions, may reflect 
different community standards or values. This is perhaps best seen in the arena of child abuse and neglect. 
Over the past three decades, tolerance for certain parental behaviors has declined dramatically. Spanking and 
other forms of punishment that may have been viewed as standard practice in the 1960s may now qualify as 
child abuse. In looking at trends over time, or in comparing jurisdictions, difference in community values may 
distort measures of the underlying conditions of children. 

Other indicators may reflect changes in the stigma or connotation of simation or an incident. This may 
be true, for example, of dropping out of school, teen fertility, and welfare participation rates. It can be argued 
that a high school diploma does not have the same meaning it had two decades ago. Welfare receipt, though it 
is often seen as a proxy for dependency, is subject to changing stigmas. These rates may increase or decrease 
because the stigmas rather than because of changes in child well-being. 

Reporters' perceptions of consequences . Reporters may also choose to report a condition based on 
their perceptions of what will hiq>pen when they report. Teachers may not want to call attention to children 
with problems, because it may reflect poorly on their skills. Potential reporters, st .h as teachers, may view the 
child welfare system as anti-family or Kafka-esque. Potential reporters may not report a condition because they 
believe the hairnfiil consequences~for themselves and for the families-may outweigh any benefit. They may 
prefer to work directly with the family to overcome the problem rather th^ involve the family in the child 
protection system. 
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Clarity of direction 

Second, trends in an indicator should generally represent, unambiguously, whether chUdren's conditions 
are improving. It should be clear when an indicator moves in a particular direction that it represents an 
i'oiprovement (or deterioration) in chUdren's well-being overall. Likewise, if comparisons are nude across 
jurisdictions at a point in time, a higher (or lower) value should show unambiguously whether children s 
conditions arc relatively better (or worse) in the various jurisdictions. An exception to this general proposition 
is indicators that provide contextual cues for more direct indicator measures. We may not know whether a 
change in racial composition or maternal employment rates is better or worse for children, but it may be an 
important control variable for other more direct measures of weU-being. 

There are two reasons why an indicator may not have a clear direction. First, a change in direction 
with one indicator may mask changes in the opposite direction of other closely associated indicators. Second, 
an indicator may reach a point of dhninished returns, where fiirther movement in one direction no longer 
represents an inq)rovement for children. 

M«»fin gopno5ite trends . A change in an indicator may disguise movement in another factor in 
children's lives. Without looking at both the indicator and other statistics, what appears to be a clear 
improvement in children's lives may in fact be a deterioration. Generally, this problem occurs at two connfted 
3its in the causal chain, where children can be moved (through varied reported, diagnosis, or treatment) from 
one category to another. For example, some policy makers and advocates believe that reductions m the 
substitiite care population are desirable. Growth in the rate of placement into substiwte care is viewed as a sign 
of chUdren's worsening conditions. However, without also looking at other indicators-such as the rate of child 
abuse and neglect-it is difficult to interpret the trend. 

Again examples other fields may be most Uluminating. Though most policy makers and advocates 
believe that reductions in infant mortality are a clear indication of improved health of chUdren, this may not be 
true Improved medical technology has made it possible to keep more chUdren alive through their first 
birthday But the quality of life of some of those children may be very poor, and they may die soon after Uieir 
first birthday. So, whUe the rate of death of children under one year of age may decrease, the morbidity of 
those children may increase, and the death rate for chUdren over age one may increase. A tracking of all these 
measures is advisable. A direct measure of expected quality of life also would resolve this masking, but that is 
neither an easy nor an inexpensive indicator to obtiun. 

The increasing divorce rate of the past three decades has been cited as a measure of the decline in chUd 
well-being A lower proportion of children now live with both parents at any given time, and a lower 
proportion of chUdren live with both parents throughout childhood. However, without fcoowing the rate of 
ChUdren harmed by living in households with high levels of conflict among the parents, it is difficult to say that 
this trend has made chUdren worse off. Though the recent trend in divorce rates may imply that children are 
worse off, evidence would be stronger if this indicator were presented with other trends m chUdren's well- 
being. (TWs is also a problem of a measure not direcfly related to children-a proxy variable rather than a chdd 
well-being variable.) 

Several indicators of chUdrcn's psycho-social well-being have this problem. For example, 
improvements in the kindergarten retention rate (that is, fewer chUdren being retained) implies ^t chUdren are 
d^g better. However, without looking at a number of other indicators (such as whether the chUdien not 
retained are successful in later school years or score better on tests of achievement), it is difficult to mterpret 
whether a change in the retention rate in fact shows that children are doing better. Fewer chUdren may be 
retained at that point in their school careers, only to find that they are retained in later yean or eventiially drop 
out from the frustration of being behind their peers. 

Pnimnfrfiminishingremms . Somc indicators may show improvements in chUdren's lives over certain 
parts of SSCZSlpoint of diminishing rewms, where further movement m that direction leads 
Ktle or no improvemem in children's well-being. This concept is difficult to prove with the mdicators used 
today, but with some indicators it is possible to conceive of such a point. 

O r-. 
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For exaiiq>le, there may be a percentage of children in the population who would benefit firom 
alternatives to high school, such as vocational opportunities. It is possible that a decreasing high school dropout 
rate suggests improvement in children's lives only up to that percentage. After that point, further decreases in 
the percent of youths dropping out of school reflects little iiiq)rovement in youths' general condition. The 
number in school may then contain children who would be better off elsewhere. 

This issue suggests the need for a better understanding of the costs and benefits of society's atten^ts to 
"move" an indicator. With most indicators, naive users may assume that in^rovement at any point on the scale 
has the same cost and benefit. Yet, with many indicators, at some point the costs of inq)rovemcnt may greatly 
exceed the benefits. 

Again, the exan?)le of infant mortality may best illustrate the point. Reducing infant mortality may 
come with some great expense, saving the lives of children who have high health care costs and poor quality of 
life. This creates a difficult moral issue. If one can create the correct scale to weigh such things (admittedly no 
easy task), the costs to society may outweigh the lives saved. Another exan^le is the unemployment rate. 
Some economic theorists believe that pushing enq>loyment below some point may have the negative consequence 
of raising inflation (U.S.DHEW, 1973). 

False positives and false negatives 

Third, indicators should balance the costs of false positives and false negatives. Many indicators 
appear to tell us something inqwrtaat if we assume that the appropriate population has been captured in the 
indicator. However, as with statistical hypothesis testing, two types of errors may occur. Children may be 
assessed as having a condition when their true status is not-having the condition (false positives or Type II 
error). Or children may be assessed as not-having a condition when, in fact, they do ^vc it (false negatives or 
Type I error). Errors of either type can lead to misinterpretation of an indicator. Up to this point we have 
discussed overt Type I and Type II errors, in which the incentives of people lead them to misreport. Here we 
discuss unknown and unintentional errors. 

Table 1 lists several indicators with this problem. For exan^)le, the rate of children in substimte care 
is often viewed as a proxy for children with severely abusive and ne^ectful parents. The increase in the rate of 
children in substimte care is generally interpreted as an indicator of a decline in the well-being of children. But, 
we do not know if we have the "right" children in substitute care. There may be false positives (children in 
care who do not need to be) and false negatives (children not in care who need to be). Workers in child 
welfare systems do not know who these children are. If they did, they would correct the mistakes. The 
mistakes arise partly ftom training and individual skills of the workers and partly from the unreliability of the 
measure-tbe poor "state of the art," 

A study by Peter Rossi, John Schuerman, and Stephen Budde (1994) shows how difficult it is to avoid 
(or even identify) the false positives and false negatives in the diild welfare arena. They asked several experts 
in child welfare to read case studies and indicate whether they believed the child in the case study should be 
placed into substimte care. They found little agreement among the experts on the sample of cases. The inter- 
observer reliability was quite low. This study also points to another problem as well: the lack of agreement 
among professionals about what constitutes the problem. It is difficult for professionals to agree on when child 
well-being is so threatened that substimte care is a better alternative. 

Several other indicators have similar problems, such as juveniles committed to sute custody, the 
percent of children in special education, rate of children retained in kindergarten, and the rate of reported or 
confirmed abuse and neglect. In each of this cases, false positives and false negatives are likely. Variation in 
practice and difference in definitions exacerbates the problem of determining whether child well-being is 
changing over time or determining whether jurisdictions are better or worse than one another. Using any pf 
these indicators to inq)ly something about the well-betog of children is problematic. 
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Clear interpretation 

Fourth, the reasons for indicator improvement or deterioration should be interpretable without raising 
confusion about the trends or issues behind the indicator. There are two issues here. First, one might thuik 
that, if possible, indicators should be give clear inferences about an appropriate action to take. We note above 
that linking indicators to policy is seldom straightforward. Yet, some possible causes and consequences of 
trends or differences should be clear to the audience in mind. 

Second, indicators should not combine two separate aspects of children's lives unless there is a clear 
rationale. One exan5)le of such an indicator ftom Table 1 is the percent of teens not in school and not in labor 
force. A decrease in the percentage of young people not in school and not in the labor force may be related to 
changes in the number of young people pushed out of school, leaving school voluntarily, refusing existing jobs, 
not being able to find jobs, or all of the above. When this indicator is not broken out or displayed with other 
indicators-and without a clear understanding of causal relationships within and between the education system 
and the labor market-it is difficult to make sense of it. Such an indicator can do little more than pique the 
curiosity of the reader. 



General Propositions about Indicators 

Having outlined particulars about pycho-social indicators, we move here to outline general propositions, 
looking at two levels: (1) the set of indicators and (2) the single indicator. At the set level, guiding principles 
include con^rehensiveness, balance, and fairness. With individual measures, we worry about reliability and 
validity. We also return to our concerns about using indicators to "evaluate" public policies. 

A set of indicators 

Balanced reporting. Our first concern is balanced reporting. Patton (1986) discusses balanced 
reporting in the following manner: 

[It] means considering multiple possible interpretations and analyzing various kinds of indicators. 
Single indicators are seldom adequate in the search for balance. Multiple indicators, multiple analytical 
2q)proaches, and multiple perspectives are required for balance. Balance positive and negative findings 
as appropriate. Be clear about definitions. Make conq>arisons carefully and ^propriately. 

We believe a set of indicators should relate individual variables to each other and to a larger theoretical model. 
Our current state of knowledge makes this difficult, but we should strive towards the devel(^ment of clearer 
models that link the various domains of children's lives. 

Potential for misinterpretation . A second concern is potential for misinterpretation. Social scientists 
can never prevent indicators from being misused, abused, or manipulated (Rainwater and Yancey, 1%7). We 
can, however, try to minimize unintended or ignorant misuse. We can try to inform the general public and 
media about what indicators might mean and what they do not. We can strive for an tqipropriate balance 
between simple and complex indicators. We can try to keep lay-people and the agenda-setters in mind. 

No dau can speak for themselves. Therefore, interpretations always should accompany indicators. 
These interpretotions should include possible counter-interpreutions or rebuttals. In early suges indicators may 
be more opaque than transparent to the lay-public, but the media and public education programs may lead to 
broader awareness and acceptance of difficult concepts. 

This leads us to the following propositions about a set of indicators, which borrow fipm and build upon 
the list presented by Moore (1994). These six propositions are value judgements on our part, the product of our 
traimng as social scientists. They no doubt would not satisfy the needs of advocate in many cases. 
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First, we would like to see a set of indicators include a comprehensive array of variables that covers: 

• the primary domains of children's lives 

• during inq)ortant age-appropriate developmental stages that speak to 

• potential areas of national policy and interest and 

• have a high chance of anticipating the chaotic, political nature of agenda setting. 

Second, to encourage a balanced view of U.S. children and their prospects, we would like to sec a set 
of indicators thitt includes both positive and negative outcomes. Negative indicators may generate more calls for 
action-bad news sells more ncwspapers-but it may focus too much on negative conditions that generate national 
pessimism, cynicism, and inaction, in our opinion. Furthermore, positive indicators may better capture 
preventive efforts, which may be missed by negative measures alone (Garbarino, 1991). 

Third, a set of indicators should report data that compare geognq>hic areas in a context and form in 
which readers can reasonably sense the relative risks and outcomes for children in those areas. We agree with 
zm (1991) that, if states are to be compared, state rankings should be replaced by calculations of whether states 
are performing better or worse than expected. (Of course, the statistical model for "leveling the playing field* 
should be made transparent.) Alternatively, dau can be presented in cross-tabular fashion using major variables 
known to make a difference-such as parents' education, income and race-ethnicity (Zill, 1991)-allowing an 
informed reader to adjust the rankings himself or herself. 

Fourth, a set of indicators should include indicators that are relatively uncorrclated. Each indicator 
should add information, though some overlq) may be necessary to assure balance and confirm relationships. 
The more the set follows some general theory of causality, the less one needs to worry about these correlations. 

Fifth, a set of indicators should present an even-handed and fair picture so that a reader docs not reach 
unintended judgements or conclusions. No set of indicators presents "the truth." There arc multiple realities, 
perspectives, interests, and values. Discussions of indicators should incliide alternative interpretations. 

Sixth, a set of indicators should be timely and periodic. As we have mentioned above, "timely" is 
difficult to anticipate because of the unpredictable nature of agenda setting. Also, "periodic" is more expensive 
than "occasionally." We believe, however, that the benefits of timely and ftequent data collection are great. 

Special condit ions in using indicators for evaluation 

In our view of indicators as dau to inform the public debate, indicators arc more like early warning 
signals-canaries in the mines or swallows in the spring-not designed for evaluation. Since the GPRA, 
according to our interpretation of the current discussion, intends to design indicators to evaluate programs, we 
present special conditions that seem to us might apply in such situations. 

Indicators could be used to evaluate programs or hold officials accountable when the following 



A program goal has been articulated in measurable terms. 

An official has accepted responsibility for achievement of that goal or h.^ been designated 
responsible by executive or legislative direction. Furthermore, she/he has control of most 
(all?) of various government programs associated with the "treauncnt." 

A program treatment or set of activities can be defined and measured. Some sense of the 
relationships among client attributes, treatment, and outcomes can be articulated. 

An appropriate research design and counterfactual (what would have happened in absence of 
the program) can be demonstrated. 



conditions hold: 
(1) 
(2) 

(3) 
(4) 
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(5) A rough cost-benefit (in a broad sense including quantitative and qualitative dimensions) 
analysis can be calculated. 

(6) An array of external factors that may contribute to the outcome arc known and controlled. 

We believe this is a reasonable test, but one that indicators as evaluation instruments will seldom pass. Some, if 
not many may disagree and believe this is far too conservative a test for indicators to pass. We solicit opmions 
concerning whether this is a serious issue and. if so. whether such conservative standards arc justified. 



f -i^ pclusions and py^^j^mniMirfations 

In conclusion, we believe is it important to continue to improve our measures of the psycho-social well- 
being of children, applying social science to improve advocacy indicators. Indicators of children's p^cho-social 
well-being can. in some cases, help define policy problems and help redefine the pubUc agenda, but they also 
can be misused by advocates and misunderstood by policymakers. 

TTuoughout this paper we have critiqued several indicators drawn from service system administrative 
dau. We have not offered these critiques because we believe surveys should be used to create all mdicators of - 
psycho-social well-being. Rather, we believe indicators from social service daw must be used, should be used, 
and can be made better. We offer two suggestions for improvement. First, we believe indicators arc best when 
used together in ways that present several sides of the same problem. This is similar to Patton's (1986) 
recommendation for a multi-disciplinary approach that considers multiple variables, paradigms, and 
interpretations. Second, we believe we can create better quality assurance mechanisms for mdicators drawn 
from the service systems. Quality assurance checks can measure whether certain types of errors m an mdicator 
are becoming more or less prevalent. This can help us understand, and possibly reduce, both mtentional and 
unintentional errors when we foUow this indicator across time and place, thus improving our ability to mterpret 
indicator trends and differences. 

Finally, we would like suggest the indicators we see as the best in this field at this time. We suggest 
the following indicators: 

• Juvenile violent crime arrest rate 

• Death rate from child abuse and neglect 

• Youth unenqiloyment rate 

• Rate of child poverty 

• Birth rates for single teens 

Each of these indicators has its problems. However, we believe that each avoids the most common and most 
serious flaws for indicators in this field. 

The f .s: two display the problems of service system mediation. Police and chUd proteaion personnel 
make mistakes in identifying (or not identifying) chUdren. However, by focusing on the most senous problems- 
-violent crimes and death from abuse or neglect~we believe the indicators minimize those problems. The 
chances for Type I and Type II errors are lessened by picking the more severe problems. 

The third indicator-the youth unemployment rate-also exhibits numerous problems. However, by 
relying on surveys of youth and letting them identify their presence in the labor market and their work activity, 
it overcomes the problems of service system mediation. 

The final two indicaton are not. strictly speakmg. indicators of children with psycho-social sittiation. 
However, we believe we must track the social context of children. Poverty rates and rates of births to teenage 
mothers are clearly related to children's psycho-social problems. 
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TABLE 1 

Exaiiq)les of Indicators of Oiildren's 
Psycho-social Well-being Now in Use 

Delinquency and crime 

1. Juvenile ancst rate (CT 94) (WI 94) 

2. JuvenUe violent crime anest rate (NDB 94; AR 93; AL 93; SD 93; Wl 93) 

3. Juveniles committed to state custody (G A 93) 

4. Juvenile delinquents in secure institutions (and average days of stoy) (WI 94) 

5. Number of homicides and firearm homicides conmiitted by Adolescents ages 10-19 (CO 93) 

6. Juvenile violent crime court referral rates (cases adjudicated per 100.000 ages 10-17 (AL 93) 

7. Number of child admissions to adult jail facilities (SD 93) 

Work and School 

8. Percent of teens not in school and not in labor force (NDB 94; AL 94) 

9. Percent graduating from high school on time (NDB 94) 

10. Youth unemployment rate 

1 1 . Percent of children in special education (TN 92) 

12. Rate of child retained in kindergarten (GA 93) 

Substance Abuse 

13. Drug use by juvenile arrestees (DC 93) 

14. Rate of substance use by high school students in the past month (CT 94) 
Child Welfare 

15. Rate of children in substitute care (CO 93) 

16. Rate of children entering substitute care over time (IL 93) 

17. Number/rate of families investigated for CA/N (CT 94) 

18. Number/rate of "confirmed" reports (by type of CA/N) (AR 93) 

19. Death rate from CA/N (CO 93) 
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APPENDIX A 
Sdentinc Tests for Indicators of WeU-Bdng 

Individual indicators should, in our opinion, be put to the following tests: 
a. Scientific/technical: 

i. Reliability/accuracy 

-measure what they purport to measure (instrumentation consistent with definition and 
assures consistent inter-observer interpretation), 
-replicable by other observers, 

-conq>arable across time and areas (i.e., can be aggregated or disaggregated and 
measures do not change across time) and meaningful across demographic sub-groups, 
and 

-low measurement or sanq>ling error. 

ii. Valid 

-face (makes sense to lay-people), 

-construct (relates indicator to theory or major policy), 

-internal (variation can be correctly related to appropriate variables and causes and 

not to spurious sources), 

o Note, this is a very tough criteria. It means that one can "level the playing 
field," that is control for demographic and environmental factors which may 
be driving the indicators, and can link results to program interventions. One 
may want to relax this standard considerably or even say not appropriate for 
indicators, but only for evaluations, 
-external (can extrapolate outside data support to other areas or time periods), 
-predictive (can be related to other more in^rtant outcomes, which is particularly 
important for intermediate indicators which are only proximate to the ultimate well- 
being outcome desired), 

-protects against attrition and migration errors, and 

-unit of aggregation, usually areal, is appropriate for the program treatment or policy 
application. 

iii. Feasible 

-is measurable on a timely, periodic basis 

iv. Useful 

-comprehensible and understandable, 
-informs the public debate and policy making, 
-empirical, not value judgement, and 

.^^ n iinatnhiroouslv be intemreted to tell w hethci children are better or worse off. 



V. Appropriate comparisons are possible 
-•counterfactual, 
-normative standard/target, 
-time series, and 
—among sub-groups. 

vi. Can separate the child variable from the service system delivery system response, as 
discussed in this report 
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vii. Minimizes Hawthorn and teaching-to-the-test scenarios. (On the one hand we want to 
minimize teaching-to-the-test; on the other, once an indicator has become the subject of a 
national policy standard-such as infant moitality-we want people to change behaviors to ad^t 
to the policy.) 

viii. Be able to provide both cross-sectional and dynamic longitudinal perspectives. For 
example, welfare dependency requires being able to catalog length of time by cohorts of 
recipients. 
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INTRODUCTION 



This paper is intended to familiarize conference participants with the variety of 
indicators of children's well-being currently in use that are based on federal data. The 
indicators reviewed were culled from existing government and private publications that feature 
descriptive measures of children's well-being. As such, they do not exhaust ail of the 
important measures of child well-being that are available from the vast federal statistical 
system, nor do they tap the range of measures that could be created. Rather, the collection 
represents a listing of available measures that one or more organizations thought important 
enough to publish as part of a collection of indicators of child well-beiog. 

In vompiling this collection of indicators, I have not used a rigid definition of 
"indicator. " The only hard and fast criteria used is that the data on which the measure is 
based must be gathered on a regular basis so that trends can be tracked over time. This has left 
by the wayside many valuable measures of child well-being that appear in one-time surveys 
sponsored by the federal government. Though there are many desirable characteristics that an 
indicator should have (see Moore, 1994 for a discussion), a measure must at a minimum be 
measured at regular intervals if it is to function as a social indicator. 

The review has been organized to correspond to the following conference session 
topics: health; education; economic security; population, tamily and neighborhood; and social 
development and problem behaviors. For each of the conference session topics, indicators and 
their characteristics are listed and summarized in separate tables which appear at the end of the 
paper.* The characteristics covered in the tables include a description of the variable, the age 
groups for which they are available, their periodicity, the geographic levels at which the 
indicators can be produced (i.e. national, state, and local), and the data source from which the 
indicator is constructed. 

In the text itself the following items are discussed for each of the five session topic 
areas: the major data sources from which most of the indicators are constructed; a brief 
description of the indicators themselves; and a brief discussion of any obvious limitations of 
the existing set of indicators in each area. Where appropriate, new developments within the 
federal statistical system that may address one or more of these limitations are also identified. 



' It is recommended that conference participants bring a copy of this paper to the conference so that the 
tables may be used for reference purposes. 
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CHILD HEALTH 

Sources of Data 

Data related to child health are relatively detailed and abundant. There are three federal 
sources of data that offer periodic measures of child health in the federal statistical system. 
The Vital Statistics System offers data from birth, death, and disease registries. Most of these 
data are available on an annual basis down to the county level for the entire United States. 

The National Health Interview Survey (NHIS) is a large survey (approximately 47,000 
families in 1991) which provides some measures related to child health annually; of particular 
interest is the special child health supplement offering detailed children's data approximately 
every six years. Estimates from the NHIS are limited to the national level. The NHIS is 
currently undergoing a total redesign which will be fielded beginning in 1996. 

The third major source of data for indicators of child health comes from Center for 
Disease Control's Youth Risk Behavior Surveillance System (YRBS). The YRBS is a biennial 
survey of students in grades 9 through 12. It gathers a great deal of unique data on teen 
behaviors, practices and attitudes on a range of important topics, especially health-related 
behaviors. There is a national YRBS survey, and separate surveys taken in 43 states and 18 
major U.S. cities. The state and city surveys are run by state and local government entities. At 
present, about two-thirds of the state and city surveys are based on representative samples of 
their populations. 

These data sources allow for separate reporting of most indicators for narrow age 
groups, and for the larger race/ethnicity groups (white, black, and Hispanic). In addition, 
indicators from the Vital Statistics system can be produced for Asian/Pacific Islanders and 
Native American populations. 

Table I lists child health and related indicators that have been derived from federal data 
sources and which appear in one or more publications of indicators reviewed for this paper. 
The indicators have been sorted into four categories: mortality, health conditions, health care, 
and health-related behaviors. 

Mortality 

Simple mortality rates are often reported for all children under age 18, and for 5-year 
age groups. In addition, within the first year of life, rates are commonly reported for neonatal 
(first 28 days following birth) and post-neonatal groups. 

Mortality indicators are also commonly reported by major causes of death. Common 
causes of death reported for infants and young children include SIDS, congenital anomaly, and 
unintentional injury. For adolescents, rates of "violent death" (a combination of car accident. 
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homicide, and suicide) are most often reported, though some publications also report mortality 
by type of violent death. Because most of the mortality indicators are taken from the death 
registries of the Vital Statistics system, they are available at national, state, and local levels on 
an annual basis. 

Hftalth Conditions 

A number of health condition indicators are reported for newborns, due in large part to 
the abundance of information available from birth certificates.* Two composite indicators 
based on birth certificate data, the healthy birth index and the children's health index, have 
been reported in recent national Kids Count reports. (Stt Table I for definitions) Other 
indicators taken from tiiis data source include low birthweight (< 5.5 lbs.), very low 
birthweight (< 3.3 lbs), and whether the child was bom with a congenital anomaly. 

Health conditions commonly reported as indicators for children of all ages include 
HIV/AIDS, child abuse, measles, children limited by chronic health conditions, and 
developmental delay. Among adolescents, reported health indicators include rates of venereal 
disease (syphUis, gonorrhea), whether the youth has seriously contemplated suicide within the 
last year, and rates of victimization from violent crime. 

Health Care 

Indicators of children's health care receipt include the presence or absence and type 
(i.e. public, private) of health insurance coverage, tiie proportion of children lackmg a usual 
source of care, the number of physician and dental visits witiiin the past year, and rates of late 
or no prenatal care receipt. Most of these measures are available on an annual or semi-annual 
basis for all age groups of children. With the exception of prenatal care measures, health care 
indicators are limited primarily to data at the national level.^ 

Health-Rel ated Behaviors 

Indicators of healtii-related behaviors among adolescents are relatively abundant, and 
include the proportion of teens who are sexually active, the proportion of sexually active teens 
using varied methods of contraception, cigarette smoking, problem drinking, use of illegal 



^ Birth certificate forms have, with minor exceptions, been standardized across the 50 states and the District 
of Columbia. Differences in data quality across sutes do remain, however. 

' State-level data concerning chUdren's health insurance coverage have been generated by combining five 
consecutive years of dato from the March Current Population Survey (CPS). While this method produces 
acceptable stote estimates (PoUard and Riche, 1994), standard errors can be large for the smallest states. In 
addition, for purposes of tracking trends over time, truly independent estimates can be made only every five 
years using this method. 



44 



41 



drags, riding with a drank driver, and use of a bicycle helmet/ With the exception of sexual 
activity and contraceptive use measures, which are available through the National Survey of 
Family Growth, these measures of health-related behaviors are available only through the 
YRBS, which provides mformation only for teens who are attending school. 

Regularly reported indicators of health-related behaviors for children prior to 
adolescence have been limited to whether the child regularly uses a seat belt. This measure is 
available approximately every six years through periodic supplements to the National Health 
Interview Survey. 

M qjnr T jmit atinns of Existing Health Indicators 

There are a number of obvious limitations to the existing set of indicators of child 
health. First, most indicators of health care receipt among children are not available below the 
national level from a federal data source. The exceptions are prenatal care receipt, which is 
available at the state and local levels, and health insurance coverage rates, for which state 
estimates can be produced using multiple years of CPS data. Such information ?.i the state and 
local level is of great importance, particularly for those who wish to design and/or evaluate 
state-level health care reforms. 

Second, indicators of child health-related behaviors appear to be limited primarily to 
teens attending school. This is not done by choice, but is rather the result of the limitations of 
available data. Such measures have been included for pre-teens and for teens who are no 
longer in school on a one time basis in other federal surveys.^ There is clearly a need for such 
measures to be collected on a regular basis for children of all ages and statuses. 

Third, institutionalized children are under-represented in the current stable of child 
health indicators. Of the major sources of health indicator data identified above, only the Vital 
Statistics system regularly collects data on institutionalized children. This is a significant 
limitation, since children are commonly institutionalized for health-related reasons. 

Finally, there is a lack of reliable immunization data for prr school children. The 
only regularly reported data currently available on the immunization status of 2 year olds is 
retrospective data based on parent report taken at the time of school entry. Such data are 



* Indicators related to cigarette smoldng, alcohol consiwnption and substance abuse are listed in Table V, 
Social Development and Problem Behaviors. 

' For example, the National Health Interview Survey included the full battery of questions from the Youth 
Risk Behavior Survey in the NHIS in 1992, providing dau on a nationally represenutive sample of teens 
regardless of their student sutus. There are at present no plans to repeat this survey supplement, however. In 
addition the Adolescent Health Survey, a one-time survey scheduled to be fielded in 1995, wUl provide rich 
health data on all adolescents. 
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widely considered to be inaccurate. The Centers for Disease Control has begun a 50-state 
survey that will provide detailed immunization data for two year olds for each of the next three 
years. At present, however, there are no plans to extend the survey beyond this three- year 
time period. 



EDUCATION 

Sources of Data 

Three federal databases provide data for most of the indicators related to children's 
education. The National Assessment of Educational Progress (NAEP) is a biennial survey 
measuring the educational achievement and related behaviors of children in the 4th, 8th and 12 
grades. Surveys are produced every other year which are representative of students in public 
and private schools at the national level and for each of the 41 states that have volunteered to 
participate in the program. The national assessment has been conducted since 1%9, and the 
state assessments since 1990. Reading and math skills are assessed every two years. Skills in 
other areas including science, writing, history, and geography are assessed every four to six 
years. ^ 

The National Household Education Survey (NHES) is a nationally representative 
biennial survey with a rotating set of topical modules that are repeated periodically. These 
include modules on school readiness, school safety and discipline, early childhood program 
participation, parental involvement, citizenship and civic participation. The survey began in 
1991. To date, only the first two modules have been fielded. The original survey design 
called for an annual survey with topical modules repeated every three years. Since the switch 
to the biennial design, it has not been determined how often the modules will be repeated. 

The third major source of educational data from which indicators are commonly 
derived is the School Enroltaient Supplement to the Cunent Population Survey (CPS), fielded 
annually in October. This nationally representative survey provides data on current and recent 
enrolhnent status, highest grade completed, diplomas received, and type of school. Schooling 
data are gathered on all persons ages three and over in each household. The survey, begun in 
1946, represents our best source of long-term trend data in this area. 

The Schools and Staffing Survey (SASS) is a survey of schools, teachers, and 
administrators. It is fielded every three to four years, and provides samples that are 
representative for the nation and for each state. Each sample school provides aggregate data 
of the demographic characteristics of their student population in addition to detailed 



*The National Assessment Goals Panel is currently meeting to consider changes in the frequency with which 
proficiency in all areas covered by the NAEP are assessed. 
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information on school programs, finances, staff characteristics, and other social characteristics 
of the school. Its potential as a great source of data for indicators relating to the qualities of 
children's school environment is only beginning to be explored. 

Table n lists indicators of educational well-being that have been derived from federal 
data sources and which appear in one or more of the publications of indicators reviewed for 
this paper. The indicators have been divided into four categories: enrolhnent, 
achievement/proficiency, education-related behaviors and characteristics, and school 
characteristics. 

Fnrnllment 

Among pre-kindergarten children, enrolhnent indicators include rates of pre-school 
enrollment among all such children ages three and over, and rates of Head Start enrollment 
among the eligible population. In the middle years, the focus changes to children who have 
repeated a grade, or who are behind grade-for-age. Among teens and post-teen youth, 
mdicators focus on rates of dropout and on-time graduation. 

Achievement/Proficiency 

Most of the indicators of scholastic achievement come from the NAEP, and are 
available for children in the 4th, 8th and 12th grades. Children are rated according to then: 
level of accomplishment in each of the subject areas covered, including math, reading, writing, 
science, history and geography. Reported levels of achievement include below basic, basic, 
proficient, and advanced understandmg. The NAEP reports feature as indicators the proportion 
of students in each subject area who are proficient or advanced. It is an intentionally stringent 
standard: in 1992, only 25 percent of all eighth graders taking the math exam scored at or 
above the proficient level. Separate indicators are reported by sex and for major race/ethnicity 
groups, including whites, blacks, Hispanics, Asian/Pacific Islanders, and Native Americans. 

In addition, each year many college-bound high school smdents take the Scholastic 
Aptitude Test (SAT). Scores are available separately for the verbal and math sections of the 
exam. Average scores are available for national, state, and local areas. Scores are often 
reported by sex, and for the major race/ethnicity groups. 

Rducation-R elateH Behaviors and Characteristics 

The NHES provides a number of indicators of school readiness among pre-kindergarten 
and kindergarten children, including the proportion who are read to daily, are regularly told 
stories in the household, are taken to the library one or more tunes per month, engage in art or 
music activities with an adult household member, or who engaged in other learning activities 
such as a concert, museum visit, zoo, household chores, or a discussion of family history. 
Among three to seven year-olds, the NHES also determines the proportion who have ever had 
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learning disabilities. 

For older children (grades four, eight.and twelve), the NAEP provides data from which 
a number of positive and negative indicators of education-related behavior have been 
calculated, including the proportion who average a particular number (one or more, two or 
more) of hours of homework per night, read ten or more pages per day, watch six or naore 
hours of television per day, or were absent from school three or more days in the previous 
month. In addition, NAEP provides data for determining the proportion of students in the 
three age groups who have positive general attitudes towards mathematics and science. 

School Characteristics 

A commonly reported indicator of children's school environments is the average annual 
expenditure per student. Though the federal government does collect and publish a great deal 
of information about the characteristics of schools, little of this information has made its way 
into publication efforts featuring indicators of child well-being. RecenUy, however, the Annie 
E. Casey Foundation's Kids Count project has sponsored the development of an innovative 
indicator of school environments reflecting the proportion of children attending "troubled 
schools." These are defined as schools with significant student behavioral, conduct, or staff 
morale problems as reported by school administrators in the Schools and Staffing Survey. 
This is a valuable source of data on the school environments experienced by children, one 
from which additional valuable indicators could potentially be developed. 

Major T. imit ations of Existinp Fducatioti Ttidicators 

The federal statistical system currently supports a large and comprehensive collection 
of periodically measured indicators of children's educational well-being. In addition, new 
topical modules to be fielded periodically as part of the NHES (early childhood program 
participation, parental involvement, citizenship and civic participation modules) will expand 
that coverage in new and informative directions.' 

One significant weakness in the current set of education indicators is the lack of 
measures on the quality of children's school environments. The Schools and Staffing Survey 
offers an existmg source from which additional indicators of this sort may be constructed. In 
addition, the NHES early childhood program module may provide important information data 
regarding the quality of pre-school environments. 



7 In addition date to br collected in the Early Childhood Longitudinal Survey, a one-time special survey 
sponsored by the Departmc at of Education, wUl also meet the need for better date on school readiness and 
achievement among young children. 
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ECONOMIC SECURITY 



Sources of Data 

Four federal surveys provide data for most of the indicators related to children's 
economic security. The Decennial Census provides information on income receipt, 
enq)loyment and housing quality. Though this information is not as detailed as in the other 
surveys, the Census is unique in providing the capability to produce estimates for small 
geographic areas (down to the block group) and for relatively small population subgroups (e.g. 
Native Americans) that are not as well represented in the smaller and more frequently fielded 
federal surveys. 

The Income and Demographic Supplement of the Current Population Survey (CPS), 
fielded each March, is a large annual cross-sectional survey of the U.S. non-institutionalized 
population involving approximately S7,(XX) households. National-level estimates can be 
produced separately for narrowly defmed age groups and major race/ethnicity subgroups with 
this database. State-level indicators can often be produced by combming three to five 
consecutive years of data, although such estimates usually cannot be produced for age or race 
subgroups, and the estimates for smaller states have large standard errors (Pollard and Riche, 
1994). A special CPS supplement on child support is fielded in April of every other year, 
providing detailed data on child support arrangements and receipt. 

The Survey of Income and Program Participation (SIPP) is a continuous longitodinal 
survey. Households are currently interviewed every four months for a total of 30 months. 
Starting in 1996, the length of time each cohort is followed will increase to 52 months. 
Income, program participation and employment measures are taken on a month to month basis, 
and are often more detailed and somewhat more accurate than those taken in the CPS, whose 
measures refer to the previous year. SIPP's smaller sample size has made it less useful than 
the CPS for providing indicators for smaller population subgroups. This relative disadvantage 
will be reduced significantly starting in 1996, when the sample size for each new cohort will 
be increased to SO,(X)0 households. Special child care, child support, and child well-being 
topical modules are fielded twice for each cohort. 

The American Housing Survey is a biennial survey that monitors the quality and 
quantity of America's housing stock. In addition to the national survey, there are individual 
representative surveys of 44 major metropolitan areas. Each metropolitan area is surveyed 
once every four years. In addition to extensive information on the quality and cost of the 
physical residence and characteristics of the neighborhood, basic demographic and income data 
are gathered on the residents of each household. Thus, indicators of children's housing and 
neighborhood quality can be constructed from this data source. 

Table HI contains a listing of existing indicators of children's economic well-being that 
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are currently available through the federal statistical system. They have been sorted into four 
categories: poverty/income, income support programs, employment, and housmg. Except 
where noted on the table, all measures described are available on an amiual basis at &e 
national level. Indicators based on data firom the CPS can be produced for states by the five- 
year averaging method described above, with its attendant limitations. Indicators of economic 
Lurity below the state level are only avaUable every ten years from the Census. Virtoally all 
of the hidicators represented can be produced for age-specific subgroups, as is mdicated m 
Table HI. 

Pnvprfy/Tncome 

Existing indicators of children poverty include extreme poverty ( < 50% of the official 
poverty line) poverty, and various operationalizations of "near poverty ranging from 125 
S to20b^rS cf the poverty line. Such poverty indicators are virtually always based 
^ToSfSi^vertyTine^^^^^^^ of children's household income a^ most o^n 
expressed as median oVmean levels of annual income Finally, there are ^^^^^ f ^^^^ 
Jicators based on the receipt of income from particular sources. Most of Aese are reflecuve 
of participation m federal income support programs, which are described below. Exc^uons 
are the iiSicators related to receipt of child support, wWch include the "PJ°PO«^°J °f 
families not receivmg child support payments" and the "proportion of mother-headed fa«^^ 
receivmg child support" withm the previous year. Data on other nonfederd sornxes of mcome 
(e g einings, inv^ttient income) are widely available through the federal stausucal system, 
but live not been developed as indicators of children's economic well-bemg. 

finvprnment gnppfift T>rngrams 

Indicators related to government support programs include the proportion of chUdren 
living in famiUes who have participated in the following programs: AFDC Food Stamps, 
subsidized or public housing, energy assistance, and free or reduced price Imiches^ Typically 
there are separate indicators for each type of program, though useful composite mdicators 
reflecting participation in multiple programs or any of several programs could certamly be 
constructed. 

Fmplnvment 

These include both parental employment measures, and measures of emplojroent 
among older children. Indicators of parental employment include the proportion of children 
X^Mential parents are in the workforce, the proportion of children witti no parente 
wS ^ the proportion for whom no parents are folly employed (workmg foil time foil 
yeTi^ ZLZ reflect concerns about both family economic stability, and the absence 
of pareS/child time due to parental labor force activity . Often such mdicators are given 
Sp^te^by the age of the child, with particular attention paid to the expenence of younger 
children (less than age 1, 3, or 6). 
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Youth employment-related indicators that have been used include a straight 
unemployment rate among 16-19 year olds, and the proportion who are idle (not in school, not 
at work or in the military, and not a homemaker). These are often reported separately by 
gender. 

Housing 

Indicators of housing quality that have been produced include the proportion of 
children living in crowded conditions (< 1 room per person), living in houses with inadequate 
plumbing or kitchen facilities, living in relatively expensive housing (with housing costs 
exceeding 50% of family income), and living in housing with moderate to severe physical 
problems. 



Ma^jor imitations of Existing Indicators of Economic Security 

There are two major limitations to existing indicators of children's economic well- 
being. First, with the exception of some housing data for selected metropolitan areas, 
measures of children's economic well-being are not available between decennial Censuses 
below the state level. Even at the state level, indicators can only be produced by combining 
multiple years of CPS data, and are not available for age or race specific subgroups. This is a 
major shortcoming of the existing federal statistical system, since such information is often 
needed for economic and government program planning. In response, the Bureau of the 
Census has been looking seriously at developing a survey that would collect the information 
currently collected on the census long form for large samples of the population on a 
continuous basis. This potential data resource is being called the Continuous Measurement 
Survey (CMS), and may replace all but a few basic demographic questions on the Decennial 
Census. High quality estimates of children's economic well-being would be produced annually 
for states and for metropolitan areas with populations in excess of 250,000. Reliable estimates 
down to the census tract level would be available every five years. A pilot CMS is scheduled 
to be fielded this year. 

The second major limitation is the lack of longitudinal measures of economic well- 
being. It has been amply demonstrated that a family's income levels and program use can be 
very unstable from year to year (Duncan, Brooks-Gunn and Klebanov, 1994). Measures that 
look at multi-year poverty or long-term income support program use will identify children in 
families who are experiencing prolonged economic difficulty and/or dependence (Duncan, 
Brooks-Gunn and Klebanov, 1994). Fortunately, data already exist to construct such 
indicators. The SIPP survey currently follows respondents for two and one-half years, and will 
shortly expand to four and one-half years of coverage.* Another federally sponsored data set 



' A special SIPP cdiort, which began in 1993, will be followed for a total of ten years. 
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that has not been discussed here, the Panel Study of Income Dynamics, can produce periodic 
measures on long-term poverty and program use covering even longer periods of time. 
Finally even the Current Population Survey is capable of yielding some multi-year poverty 
and program participation indicators. Due to the nature of the CPS sampling scheme, about 
45 percent of the sample households also participate in the survey the following year. 



POPULATION, FAMILY, AND NEIGHBORHOOD CHARACTERISTICS 



Sources of Data 

The indicators included in this section are primarily descriptive, demographic 
measures. The four primary data sources for these mdicators are the Decennial Census, tl 
Current Population Survey Income and Demographic Supplement, the Survey of Income i 
Program Participation, and birth certificate data from the Vital Statistics system. Each of 
data sources has been described above. 



r\^i\t\ rhararteristics 

Indicators of basic child characteristics include the number of children, the percent 
within each major race/ethnicity group (white, black, Asian/Pacific Islander, Native American, 
and Hispanic), the percent who changed residences in the previous year,' and the percent who 
are linguistically isolated.*" Children as a percent of the total population has also been used as 
an indicator, though its relation to well-being is unclear. 



Family and ^niififfhnld Characteristics 

Common indicators related to family structure and living arrangements include the 
proportion of children living in two-parent and single-parent families, with step-parents, and 
with neither parent, in subfamilies within multi-generation households, and in instimtions or 
group headquarters. There are in addition several family indicators related to birth and family 
formation including the percent of buths to unmarried women, to teens, to unmarried teen 
women and the rate of second births to unmarried teen mothers. In addition, a composite 
indicator called the New Family Index reports the percent of first births to women with less 
than 12 years of schooling, who are unmarried and under the age of 20." It is an indicator of 



»Frequent residential moves have been associated with subsequent academic and other difficulties (Coleman. 
1988). 

'"Linguistic isolation is defined as any child age 5 or over living in a household where no person ages 14 or 
over speaks English only or very well. 
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the proportion of new families that are high risk. 

Neighhorhon d Characteristics. 

The Decennial Census provides census tract characteristics related to income and 
poverty, welfare use, employment, family structure, educational attainment, and other 
population measures. To date, two such mdicators have appeared in publications featuring 
indicators of child well-being. The first is the proportion of children living m high poverty 
areas (40+ percent). The second is the proportion livmg in "severely distressed 
neighborhoods," defined as neighborhoods with values one or more standard deviations 
beyond the mean level in at least four of the following five characteristics: poverty, female 
headed families, high school dropouts, males unattached to the labor force, and families 
receiving public assistance. 

Mq jnr T jmitatinns of Existing Indicat ors of Ponulatinn Familv and Neighborhood 

The federal statistical system provides a great deal of data from which a broad array of 
usefiil child and family demographic indicators can be constructed, and which can be presented 
for age and race/ethnicity subgroups. There are two significant limitations, however. First, 
between decennial censuses, the data do not support indicators for places below the state level, 
and state level indicators can be constructed only by combining multiple years of CPS data. 
One solution currently under consideration by the Bureau of the Census is the fielding of a 
Continuous Measurement Survey (CMS) which was described in the previous section. 

A second significant limitation is the lack of longitudinal mdicators. Such indicators 
can reflect changes in family structure or residential stability over time, which are known to be 
negatively related to child well-being (Coleman, 1988; Hetherington and Clingeir5)eel, 1992). 
The SIP? may be able to support some such indicators when it expands to four and one-half 
years of coverage per cohort. Some valuable longitudinal family indicators may require 
retrospective marriage and residential histories such as those taken in the National Survey of 
Families and Households. 



"This indicator was originally developed by Christine Nord and Nicholas Zill for the national Kids Count 
Report (Kids Count, 1993). 

"See Annie E. Casey, 1994, for details. 
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SOCIAL DEVELOPMENT AND PROBLEM BEHAVIORS 

Smircesof Data 

Indicators in this area would include age-appropriate measures of psycho-social 
development, and measures of both pro-social and anti-social or problem behaviors. The 
major federal sources of data for such indicators include the National Household Education 
Survey, the CDC's Youth Risk Behavior Surveillance System, The ChUd Health Supplement 
to the National Health Interview Survey, and Monitoring the Future. The first three data 
sources are described above. Monitoring the Future is an annual survey of a nationally 
representative sample of high school seniors. The survey focuses on questions concerning drug 
use, delinquency, crime victimization, aspirations related to schooling, work, and family 
formation, and attitodes concerning such topics as race relations and the government. 

In addition, the FBI's Uniform Crime Report system provides some data on youth 
arrests, the National Survey of Family Growth provides data on sexual and fertUity behavior 
for females aged 15-19, and the National Household Survey on Drug Use provides annual data 
on teen drug use. 

Table V lists and provides descriptive information on indicators of child social 
development and problem behaviors that are based on federal data sources and which appeared 
in one or more of the publications reviewed for this paper. 

Pro-social Behaviors and Attitodes 

Available indicators in this area have been limited primarily to the behaviors and 
attimdes of teens. Existmg indicators of pro-social behaviors include the percent of teens who 
participate in organized sports, who regularly attend church, and who do the following 
activities on a daily basis: read, see friends, perform household chores, play music, do art, or 
write. Indicators related to attitodes and beliefs include the proportion who report that their 
peers approve and support hard work and good behavior, those who hold a variety of positive 
life goals as being extremely important (e.g. success in work, good family life, strong 
frieafjhips, community involvement), and the proportion who are concerned about national 
problems such as crime, drugs, hunger and poverty, race relations, nuclear war, economics 
problems, and pollution. 

Problem t> f;havinrs and Attitodes 

Indicators of teen problem behaviors include measures of drug use, delinquency and 
violence, and sexual activity. Measures related to drug use include the proportion who 
regularly smoke cigarettes, who binge drink, and who have used various other controlled 
substances (e.g., cocaine, crack, marijuana, LSD) within the last 30 days. The proportion 
reporting peer approval for such behaviors is also reported. Indicators related to violence 
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include the proportion of teens who have carried a weapon to school, or who have been in a 
fight within the last 30 days, and the proportion of 10-17 year-olds arrested for violent crimes 
within the past year. Indicators related to sexual activity include the proportion who are 
sexually active, and the proportion who have had uiq)rotected interccj^se within the past year. 

The Behavior Problems Index (BPI) is a composite mdicator of problem behaviors 
based parent report. It is a 28-item scale developed by Nicholas Zill and James Peterson, and 
is based on the Achenbach Behavior Problems Checklist (Achenbach and Edelbrach, 1981) and 
other child behavior scales. Separate but parallel measures have been produced for children 
ages 4-11 and 12-17. The BPI has been asked in 1981 and 1988 as part of the Child Health 
Supplement to the National Health Interview Survey (NHIS). Current plans call for including 
the BPI in the core set of NHIS questions that are fielded every year beginning in 1996. 

M ^or Limitations of Exis ting Measures of Social Development and Problem Behaviors 

Of all of the categories of indicators reviewed in this paper, this is the area in which 
the most work remains to be done. The most striking limitation of the current stable of 
indicators of children's social development and problem behaviors is the lack of measures for 
children prior to their teen years. A review of existmg publications featuring indicators 
produced only one measure, the BPI. And yet, this is an area in which a great deal of 
measurement work has been done, both on direct measures of social development in early 
childhood, and on measures of family functionmg and the home environment that are known to 
affect a child's social development and well-being (Zaslow, Brown, Coiro and Blumenthal, 
1994; Love, Aber and Brooks-Gunn, 1994; Phillips and Love, 1994). Though measures of this 
sort have appeared in previous federally sponsored surveys (e.g. the National Survey of 
Children, the National Survey of Families and Households, the National Longitudinal Survey 
of Youth), until very recently none have been incorporated into surveys that are to be repeated 
on a regular basis. 

Fortunately, the National Center for Health Statistics is currently planning to 
incorporate some social development and family functioning measures into the NHIS child and 
family topical module as part of its NHIS redesign. It is scheduled to be fielded in 1997, and 
will be repeated every few years, providing the data from which indicators can be constructed 
and tracked over time. In addition, the school readiness supplement to the NHES, first fielded 
only last year but scheduled to be repeated every few years, contains a number of social 
development-related measures for children age three through seven that can serve as the basis 
for new regularly reported indicators for young children. 

A second limitation has to do with inadequate coverage of the teen population. Many of 
the current indicators of social development and problem behaviors for teens in Table V are 
based on data sources that include only those teens who are still in school. While the YRBS 
and Monitoring the Future are valuable sources of data, they do not provide information on the 
very teens who are the most likely to be experiencing difficulties in these areas: those who 
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have dropped out of school. It is important that these or other surveys, such as the NHIS, be 
expanded to gather regular measurements of this sort for all teens.'' 

Finally, the existing stable of indicators of pro-social behaviors and attitodes requires 
fiuther development. Historically, considerably more time and effort have been devoted to 
conceptualizing and tracking negative behaviors and attitodes than positive ones. (See Moore, 
1994). Both conceptual work, determining which positive and behavioral attitodes are most 
important, and further data development are needed in this area. 



" This was done on a one-time basis in 1992, when questions from the Youth Risk Behavior Survey were 
added to the National Health Interview Survey. The questions were given to a nationally representative sample 
of all children ages 12 and over, regardless of student status. 
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Under its Kids Count program, the Annie E. Casey Foundation supports both national and state projects 
on indicators of child weU-being. The first national Kids Count Data Book was published in 1990, and the 
mitial state-level Kids Count dau books appeared in 1991 and 1992. State Kids Count reports fimded by Casey 
Foundation grants arc now proceeding in 47 states plus the District of Columbia. The Casey Foundation 
expects to sign contracts with projects in the three remaining states (Alaska, Nevada, and Virginia) by the end 
of 1994. As of September 1994, projects in 35 states plus the DUtrict of Columbia had produced at least one 
annual Kids Count report under a contract with the Casey Foundation. Projects in two other states 
(Pennsylvania and Utah) had produced similar daU reports with funding from other sources. 

This intensity of reporting on the condition of U.S. children at the subnational level, largely attributable 
to the efforts of just one foundation, is unprecedented. The closest con^Muison may be the early-twcntieth- 
century city survey movement, which received support from the newly established RusseU Sage Foundation 
starting in 1906. That foundation promoted large, detailed surveys of sodal conditions in several cities, and 
also funded a renamed and reinvigorated journal of the social work establishment (called Sism «» 
1909) to report on smaller studies. The survey movement expanded rapidly with Sage Foundation support, 
influencing major pre-World War I fire prevention and sanitation reforms in many cities. Still, the surveys 
were never as geogn^hically comprehensive as is the current Kids Count effort. 

In this paper, I describe the state Kids Count reports so far produced and assess lessons they may offer 
to others interested in examining the condition or well-being of chUdren.' As part of this effort, I reviewed 
Kids Count reports ftom 21 states and the District of Columbia, plus a similar (but not Casey Foundation- 
funded) report from Pennsylvania.^ Time limitations -rented me from reading any of the reports entirely or 
from reviewing reports ftom all the states. My samp^ -g design, both within each report and among all possible 
stete reports, can most charitably be represented as purposive; I wanted to review reports from different parts of 
the country, reports that were organized in different ways, reports prepared by different kinds of agencies 
(university research cwiters, state government agencies, and advocacy groups), reports from states with many 
counties and sutes with few counties, and reports that displayed more and less narrative. I performed no 
formal contem analysis on the reports I did read; that is, I did not start with a list of qualities and measure how 
many reports contained each quality. This paper, then, represents one person's somewhat informal readmg of 
selected reports that might not be representative of all state Kids Count reports. 

Although the reading was informal, it was also, at least from one perspective, educated. I have 
prepared one (and am close to completing a second) state Kids Count dau book for the Wisconsin project, and I 
understand many of the tensions that arise in decisions about dau seleaion and presentation in these limited- 
budget reports. Despite the major funding commitment of the Casey Foundation to the projects, limiutions in 
the dau development, dau management, and writing stages can be reasonably severe when $100,000-$150,000 
per state per year-plus whatever grants can be raised from other sources-must be spread among dau 
development and management, publication design, publicity, paper and printing, and advocacy efforts. In my 
state, about half my time is all that is available for writing, dau development, and dau management. 



'This paper is, in part, an update of an earlier review by other authors of the first seven sute Kids Count 
reports. See Brett V. Brown and Nicholas Zill, "A Review of the First Seven Sute Kids Count Annual 
Reports," Washington, D.C., Child Trends, Inc., September 1992. 

^The 21 sutes are: Alabama, Arkansas, Colorado, Connecticut, Georgia, Illinois, Indiana, Iowa, Kansas, 
Louisiana, Maryland, Minnesou, Montana, New Jersey, New York, North Carolina, South Carolina, 
Tennessee, Vermont, West Virginia, and Wisconsin. 

'Many state dau books differed significanUy ftom the Wisconsin books I prepared-sometimes in ways that 
challenged my perceptions of the quality of the Wisconsin product. I have tried not to be defensive about these 
differences, or at least not to let any defensiveness affect this paper, but I may not entirely have succeeded. 
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Because my leading of the state data teports was infonaal, to are my fmdings, which I have organized 
in six topic areas: purpose, measuring the status of children, individual indicators, a frequent theme, 
con^arisons, and organization of the books. 

Purpose of the Books 

The 1993 dau book for Pennsylvania neatly summarizes the several purposes behind most of the state 
reports. This book notes (p. 5) that the document is intended to "serve at least four purposes. It provides 

1 . A centralized, accurate, and accessible dau base that contains information about the status of 
our children for policy makers, advocates, parents, media, as well as the business, labor and 
religious communities. 

2. A vehicle to raise awareness throughout the state and in every county about how our children 
are faring. 

3. A tool to assess where children's needs are greatest so that we can target available resources 
naost effectively. 

4. A baseline so that we can marie successes and failures as we woric to in^)rove the lives of our 
children." 

The various state Kids Count projects en^hasize some of these purposes more than others, with little 
control exerted by the Casey Foundation. The 1994 Louisiana data book, for exan^le, limits its advocacy to 
part of a gracefully written introduction of six pages. The introduction remmds readers that economic well- 
being, health, and sdiool performance are all connected and that African American children in Louisiana score 
less well on all numeric indicators than do white children. The introduction then says that all the indicators 
could be improved if Louisiana made work pay for welfare recipients, offered pregnant women better access to 
prenatal care, offered teens routes other than parenthood "in their search for nurture, self-esteem, and a 
connection to the fiiturc," and invested more in preventing and responding to child abuse, etc. The remainder 
of the document consists of tables which report numbers on 33 indicators for the state as a whole and for each 
of Louisiana's 64 parishes. . 

In contrast, the 1993 Iowa report devotes its first eleven pages to a plea for a "common sense program" 
for "investing in families, prevention, and sdiool readiness." This section proposes a detailed expansion, 
costmg $34 million annually, of existing Iowa programs targeted at families who could benefit from early 
intervention. The next pages of the document offer a precise description of six existing early intervention 
programs in Iowa that could benefit from the expansion fimds. For each of the six programs, a success story of 
an individual family is presented. (For exan^)le, Louise, in her fifth month of pregnancy with her fourth child, 
was facing eviction from her apartment, using cocaine, and receiving no prenatal care. Staff in an early 
intervention program found her an apartment, acconq)anied her to an obstetrics appointment, and nurtured a 
relationship that eventually included regular drug abuse treatment, immunizations for all her children, and 
anangements for a special education program for one chUd. After giving birth to a healthy baby, Louise took 
"steps to prevent the birth of another chUd" and enrolled in a parenting class.) Only the last eleven pages of the 
Iowa document-just one-third of the book-are devoted to cramped tables showing numbers on nine indicators 
for Iowa as a whole and for each of its 99 counties. 

I do not know whether the Iowa or Louisiana approach is the better strategy. With inevitable 
limitations in funding and space, choices must be made, and doubUess some factors particular to each state 
influence the choices. The Wisconsin 1994 report, for exuaple, contains even less explicit advocacy than the 
Louisiana report. Part of the reason for the neutrality in the Wisconsin daU book was a project goal of 
attracting donations of free paper for future reports from the Wisconsin paper industry. The 1993 New Jersey 
dau book is the only one I reviewed that contains no narrative, not even a brief introduction, and hence no 
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explicit advocacy. One reason may be that the Public Service Electric and Gas Company of New Jersey was a 
major fiinder of the book. 

States also have varying political and mass media traditions that may influence the nanue of the data 
books. A neutral display of numbers prominently picked up by state media may have profound intact in some 
states; in others, a specific program proposal may serve to galvanize political action. In states with a strong 
tradition of policy control at the county and school district level, or in which sources of political power are 
dispersed and often changing, a "reference book" approach to the data reports may be sensible. In these states, 
the data books primarily make available a wide variety of information in an attractive, easy-to-read format, with 
the assumption that people wishing to influence state or local policy will use the information in ways that the 
preparers of the document cannot predict. The intent of the preparers is to include as much infomuttion as 
possible, for who knows what particular bit of dau might sway some future decision? 

The type of agency preparing the dau book also seems to affect the kind of book produced in each 
state. With some exceptions (New Jersey is the most prominent), dau rqports prepared by advocacy 
organizations (for exanq)le, Jhe reports from Iowa, Indiana, and Connecticut) contain the most narrative and the 
most explicit advocacy. In general, reports prepared by university departments and institutes (such as the 
Arkansas, Louisiana, and Wisconsin dau books) tend to have more dau and less explicit advocacy. The few 
dau reports prepared by state agencies, such as the 1993 Tennessee report, also present more dau and less 
narrative and explicit advocacy. 

Of course, many preparers of the low-narrative reports might argue that the dau themselves serve as 
advocacy. The Tennessee report, for exanq»le, states (on page 4): "The goal of this report is to provide 
information to increase public awareness of the serious problems that face many children in Tennessee. It is 
hoped that this increased awareness will encourage grassroots support for public and private efforts to improve 
the quality of life for children." Still, if fimders want more or less explicit advocacy, the easiest way to exert 
influence may be through the kind of organization selected to prepare the document. 



The Usefiilness of the Dau As a Measure of the Sutus of Children 

Although the dau reports may have several purposes, including that of serving as a broad reference 
tool for state and local citizens to use in unforeseeable ways, certainly one purpose of many of the books is, as 
the Pennsylvania report suggests, to create a baseline for marking future inq)rovement or regression in the well- 
being of children. In one sense, any dau about children collected in the same way for several years advance 
this purpose. Intuitively, however, some dau do so better than others. No dau report that I reviewed showed 
the winning times over many years in the 4(X)-mcter race at the state high school track tournament, although 
presumably this information is readily available. Somehow, we know that following such dau over time tells us 
little in^rtant about the condition of children. 

But how do we know this? Nicholas Zill and others have suggested that whatever data we track can be 
conceptualized as lying in five domains of well-being: economic and material adequacy, physical health and 
safety, cognitive development and academic achievement, emotional well-being, and moral development and 
social behavior.* Because 400-meter race times could fit under the physical health and safety domain, we still 
need some way of separating important from less important indicators within the domains. Brown and Zill are 
probably correct to suggest that the key to this puzzle lies in identifying important developmental milestones 
within each of the domains and then measuring the degree to which children have achieved those milestones.' 
We do not yet, however, have consensus on the milestones and the appropriate way to measure whether they 
have been readied. 



*See. e.g., Nicholas Zill and Mary Jo Coiro, "Assessing the Condition of Children," Ctilldrcn Md Ywti> 
ServicesReview . 14 (1992), 119-136. 



'Brown and Zill, "Review of the First Seven Kids Count Annual Reports," pp. 1-2. 
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In states like Wisconsin, in which a reference book q>proach is considered desirable, probably two 
distina parts of the dau books will some day be i^ropriate. One part could provide the broad reference 
information that may be usefiil in unpredictid>le ways to a wide variety of citizens, and the other part could 
show the core variables that indicate how children are 'really' doing. At this point, we probably do not know 
what these core variables are. 

The next section of this report discusses indicators that states have so far used. Before turning to the 
individual indicators, however, it is useful to note that several state data books argue the general inqwrtance of 
concentrating on 'outcome' rather than 'input' indicators. If one wishes to measure how children are really 
faring, this kind of distinction seems i^ropriate, but it is not easy to maintain. Most data projects, for 
exanq>le, treat births to single parents as an inqwrtant indicator, but is this an 'outcome' (of the circumstances 
of young women and men) indicator, or an 'input' (to the life chances of the newborn child) indicator? Many 
other frequently selected indicaton, such as child poverty or the number of AFDC recipienu, share this dual 
characteristic of being both an outcome of some circumstances and an input to others. A simple outcome/iiq>ut 
dichotomy is thus difficult to maintain. 



Individual Indicators 

The national Kids Count Data Book for 1994 uses the following ten indicators of child well-being by 

state: 

Percentage of low-birthweight babies 

Infant mortality rate 

Death rate of children ages 1-14 

Percenuge of all births that are to single teens 

Arrest rate of juveniles for violent crimes 

Percentage of high school students graduating on time 

Percentage of teens not in school and not in the labor force 

Teen violent death rate 

Percentage of children in poverty 

Percentage of children in single-parent families 

Sute Kids Count reports frequently do not use all these indicators when reporting at the county level, 
partly owing to problems of data availability. Three indicators in the national report-percentage of teens not in 
school and not in the labor force, percentage of children in poverty, and percentage of children in single-parent 
families-are available to the Casey Foundation in the years between decennial censuses only from the annual 
March supplement to the Current Population Survey (CPS). Because the CPS does not survey enough famUies 
in any single state to allow for statewide generalization, the national Kids Count rqwrt merges the most recent 
five years of CPS dau to arrive at state-by-state numbers. This methodology may offer an i^ropriate 
estimator at the sute level (although it would be helpful if the national Kids Count dau book showed standard 
errors for each state, ptthxps in small print at the back of the book). Except for a few counties and cities with 
very large populations, the methodology does not work at the substate level. If sute projects wish to use the 
three indicators, they must either show the number from the last decennial census or develop a different 
estimating methodology. 

So far, most states have either excluded one or more of the three indicaton or have used numbers from 
the 1990 census; these decisions may change as we move further from the 1990 census. The New York Sute 
Kids Count project, however, developed iu own methodology for estimating annual child poverty rates by 
county. The methodology tests on a set of annually measurable factors that predicted with reasonable accuracy 
the census-determined 1989 child poverty rates in each county of New York State. In less densely populated 
counties, the factors that best predicted 1989 child poverty rates by New York county were the percentage of 
persons receiving food stamps, the ratio of dissolutions of marriages to new marriages, total government 
transfer payments to individuals as a percentage of total income, the percentage change in the child population 
between the two previous (1970 and 1980) censuses, the percentage of births in which the mother had no health 
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insurance or was on Medicaid, and the percentage of women receiving early prenatal care in the estimate year. 
Each of these factors was weighted, transfer payments having the largest effect on the estimate, followed by 
marriage dissolutions and then food stamp recipiency. No one knows, of course, how close this estimating 
methodology will be for child poverty by county in 1999.' 

Problems also arise in the efforts of state Kids Count projects to report high school graduation rates by 
county In the national Kids Count report, graduation rates arc defined as the number of public high school 
graduates in a state divided by the public ninth-grade enrollment four years earlier, with some adjustments for 
overall migration to or from the state. The Casey Foundation may choose to report on graduation rates, rather 
than dropout rates, in part because graduation represents positive achievement, while dropout is a measure of 
negative problems. But graduation rates are more difficult to compute than are dropout rates. No good 
estimate of net migration at the county level exists in the years between the decemiial censuses, and high schools 
in counties with high rates of net in-migration may show a graduation rate of more than 100. Some states, such 
as Arkansas, New York, West Virginia, and Wisconsin, show dropout but not graduation rates by counQ^ 
Other states, such as Colorado, show both a dropout and graduation rate. A few projects, such as the 1993 
New Jersey data book, include no information on school achievement. 

The remaining indicators in the national Kids Count bock arc more readily available for substote 
jurisdictions. The juvenUe violent crime arrest rates in the national data book stem from well-established 
uniform crime reporting procedures administeied by the Federal Bureau of Investigation. Accordmg to FBI 
standards, murder, forcible rape, robbery, and aggravated assault are considered violent cnmes. Aggregating 
reports of these crimes ftom individual law enforcement agencies to the county level is a relatively 
straightforward procedure, and many state data reports include this or similar information for each county. 
Several state dau books (for example, the Georgia and West Virginia books), however, contain no information 
on arrests, but report instead on the number of juvenile delinquents on parole or in correctional facilities. This 
exclusion of violent crime arrest data may reflect concerns that police and prosecutorial emphases can skew 
violent crime rates; the largest component of that rate, aggravated assault, cannot always be precisely 
distinguished from simple assault, which is not considered a violem crime. A few state reports, such as Iowa 
and New Jersey, contain no information by county on delinquency. 

The remaining core indicators in the national Kids Count daU book-low birthweight babies, infant 
mortality, child death rate, percenuge of all births that are to single teens, and teen violent death rate-all derive 
ftom well-established vital records systems. The information sources for these indicators arc birth and death 
certificates routinely fUlcd out in all sutes and easUy available at the county level. It is possible to be critical of 
some of these indicators. For example, the percentage of all births that arc to single teens depends as much on 
the fertility of older women and married teens as it does on the fertility of single teens. If the number of births 
to single teens were to fall, but not as fast as the number of births to women older than age 19, an improving 
situation (fewer teen births) would appear to be a worsening trend. In some fonn, however, the vital records 
data seem clearly useful, and every state Kids Count report makes substantial use of datt ftom birth and death 
certificates. 

Many states supplement the indicators appearing in the national dau book with a large number of otiier 
variables TTie most common are reports or substantiations of chUd abuse and neglect, the number or rate of 
cWldren in foster care, number of women in the labor force, participation rates or numbers in such programs as 
Food Stamps and AFDC, data from standardized school achievement tests, the percentages of pregnancies m 
which prenatal care was provided in the first trimester, average household income by county, and child 
immunization dau. (No sute included all these variables.) 



«The methodology did predict 1979 child poverty rates by New York county reasonably well. Sw Sdh 
Leon and Nancy Dmiton, |>^v>>.n«,l R.timates of Povf rtv for New York State COUntlK. New York Sute Kids 
Count, Albany, New York, May 1994. 



87 



One legacy the state Kids Count projects might leave to future generations is not just systematic 
reporting of existing datt but an investment in new sources of data on children. So far, few states appear to 
have moved in this direction, although some exceptions stand out: 

• The development in the New Yoric project of a procedure for estimating child poverty by county 
between the census years. 

• The addition of a question on concerns about the status of children to an annual siirvey of a probability 
san^le of adults in Arkansas. 

• The use in Georgia of an aggregate variable which assesses the life chances of a child at birth by 
merging three maternal charactcristics-whetherthe mother is married, whether she has a high school 
education, and whether she is over age 20.^ 

• The use in Georgia of dau from a 1990 national survey by the Centers for Disease control on "youth 
risk behavior." The Georgia project utUized state tables fipom the survey to estimate teen sexual activity 
in the state. 

Although cost will be a barrier, the state projects might consider developing new forms of data. 



A rnmmnn Thame 

The introductory nanatives of several state Kids Count reports follow a Dickensian "best of times and 
worst of times" theme.* The narratives declare that many children in the state are doing well and, overall, the 
state is doing better in terms of certain indicators now than in the past. On the other hand, many children are 
not doing so well, and many indicators show a worsening of child conditions in the state. Several quotations 
illustrate this theme: 

Colorado (1993, p. 8): "Most of Colorado's children and families flourished during the 
1980s, but a growing number did not." 

Illinois (1992, p. 4): "Many children did make their way through school, healthy, safe, and 
well-educated. We should be thankful for their successes. However, far too many children in 
every part of the state were left out along the way." 

Maryland (1993, p. 7): "Maryland's Kids Count Factbook shows that there are some 
promising improvements in child well-being. ... Yet across the state, the disparity in the 
quality of the lives of children is disconcerting." 

Vermont: (1993, p. 8): "The majority of Vermont children are healthy, live in two-parent 
homes in which both parents work, aren't poor, aren't abused, don't get into trouble with the 
law, and graduate ftom high school.... Vermont also should be a leader for children and 
families who slip through the cracks. They are the thousands of kids going to bed hunpy, the 
one in five who lives in a single-parent family, the hundreds of teens who drop out of high 
school, the hundreds each year who bear children before they are 20, and the mothers who 
must work but lack adequate child care." 



^is merged variable also appears in Nicholas Zill and Christine Winquist Nord, Ryniling in Plaw. H9w 
American F«mili»^« Am Faring in | rh.n«ni> Economy and «i Individualistic Society (Washington, D.C.: ChUd 
Trends, Inc., 1994). 

*See the opening paragraph of A T«i(> of Two Cities. 
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In any state, of course, some children will do better than others on any given set of indicators. In 
almost all sutes, it appears that some health indicators, such as infant mortality, have improved with the 
development of more sophisticated medical technologies, while other indicators have regressed. Certainly, it is 
inqmrtant for the data books to portray both inq>rovements and regressions, if only because policymakers told 
just about problems might feel so overwhelmed as to lose hope for in^rovement. It is to be hoped, however, 
that the repeated theme of some children faring well and others not can avoid becoming formulaic and trite. 

Makii^ Comparisons 

Many state reports make either geogn^hic or temporal comparisons for each indicator the report cites. 
In geographic conq>arisons, each county is compared with every other county for that indicator, and a rank 
Older is established. In ten^ral conparisons, each county's performance in a current year is conpared to a 
base year. 

The use of conq)arisons raises a few issues. First, as Brown and Zill have pointed out, an individual 
ranking for each county can suggest larger differences among counties than are actually present. The difference 
in performance between a county ranking fifth and fifteenth may actually be small; if the rankings derive from 
sanq>le dau, the differences may reflect the accident of the particular saaq>le rather than a real difference in 
underlying conditions. Most states at least publish the raw numbers as well as the ranks, and they might also 
consider showing standard errors especially for the relatively few county-by-county indicators that derive from 
sanq>le data.' 

The second issue in making comparisons is the question of «1iat represents better and what worse. 
This is particularly an issue in temporal conparisons, which suggest that, for any given indicator, the county is 
doing better or worse than it was in a base year. (Some geographic conparisons also indicate that a county is 
doing better or worse than the state average, but others simply rank the counties from highest to lowest, without 
arguing that the highest is best or worst.) These issues of better and worse arise especially in data that compare 
rates of program utilization. It is not clear, for example, that a county with more substantiated child abuse 
cases than at an earlier time is actually doing worse; it is possible that the county is doing a better job of 
publicizing reporting procedures and investigating the rqmrts it receives. Similarly, some readers may wonder 
why having a lower percentage of children on AFDC in the current year than in the base year is a mark of 
inprovement. The lower percentage may simply reflect a reduction or freeze in AFDC benefits. 

A few states, such as West Virginia and New Jersey, also display a total composite ranking for each 
county. Because neither project weighted the indicators differently, the assunption behind the composite score 
must be that each of the indicators is equally important to the overall condition of children. Most of the state 
projects were reluctant to make that assunption and did not try to develop a composite score. 



Data Book Organization 

Whether or not state Kids Count projects choose to follow the Iowa approach of including policy 
proposals in their annual documents, the projects have two broad choices for presentation of dau. They can (1) 
organize the book around indicators, listing on one or two pages all county scores on that indicator and then 
move to the next indicator; or (2) organize the book around counties, showing on one or two pages each 
county's score on all the indicators. Organizing around indicators allows for some narrative discussing the 
meaning, strengths, and weaknesses of each indicator, but does not allow a reader to see quickly how one 
county is doing on all the indicators. Organizing around counties promotes an easy reference for how a county 
is doing, but makes meaningful discussion of the indicator more difficult. 



•Brown and Zill, "Review of the First Seven State Kids Count Annual Reports," pp. 10-12. 
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Pan of the decision on which way to organize will rest on where policy regardiL\g children is most 
often decided. If state government makes most of the decisions, then organization around the indicators makes 
most sense. If county and other local governments make key decisions, then organizing the document around 
counties may be most appropriate. Some state projects combine the approaches. The New Yoric data book, for 
exanq>le, organizes the first half of the book around indicators: the second half reorganizes the same information 
to show it by county. Other state projects with the necessary financial and staff resources may find this a useful 
model. 
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ASSESSING CHILDREN'S WELL-BEING: HOW MANY AND WHICH INDICATORS, 

AND AT WHAT COST? 

Robert H. Haveman 
Univeisity of Wisconsin-Madison 



This conference has been a field day for iq?plied social researchers-sociologists, developmental 
psychologists, demographers, bean counters, and-yes-«ven economists. It was our opportunity to construct 
dream lists of indicators of children's environment, families, schools, characteristics, bdiaviors, attitudes, 
expectations, problems and accoii5>lishmcnts. We e3q)lored each of these dimensions in great detail, with 
multiple indicators for each of the categories identified and (in most cases) charatterized as crucial for fiilly 
understanding a particular aspect of children's up-bringing, attainments, or behavion. 

As an exan5)le of the convrehensivencss of the coverage that we gave to specific aspects of children's 
lives and well-being, consider the concept of the "neighborhood environment" in which children grow \sp. 
Clearly some children grow up in supportive neighborhoods with peers firom successfid families, while others 
are raised in "bad" neighborhoods with a high prevalence of high school dropouts, single-parent families, 
welfare dependency.' We were told that much work was necessary in order to construct a series of 
neighborhood indicators of children's well-being, in particular we needed to: 

identify neighborhoods as people themselves perceive them, and not as administrative units; or, 
barring this, "map locally based social interaction onto a spatial grid"; or, barring this, employ 
census tract or "block groups" as poor neighborhood proxies; 

include, in one study, several overli^ping definitions of neighborhood, so as to decide which is 
the best 

develop indicators of neighboriioods that were outcome indicators (we were given 15 such 
indicators for each neighborhood) 

develop indicators of neighboriiood "context" that describe the aspects of "community 
environment that may be inqwrtant factors in the well-being of children (we were given 20 
such indicators)^ 

develop neighborhood indicators that will permit demographic (age and gender, at a minimum) 
standardization 

develop such neighborhood dau on a longitudinal basis, to identify the community 
developmental trajectories 



'I have chosen this aspect of children's well-being because attributes of this characteristic that were 
iuggested at the conference were less demanding than those placed on, say, family circumstances and choices. 

*rhese neighboriiood dau need to be linked to individual children's characteristics and family processes 
(data that is "multilevel"); they must include both a sufficient number of neighboriioods and a sufficient number 
of families and children within each to support adequate statistical testt of hypotheses; these data must include 
"direct measures of all the community characteristics relevant to how one or more theories explain the influence 
of neighboriioods on the developing child" This will include information on: infrastructure, housing 
characteristics, conditions of homes and open space, neighboriiood institutions (police, social welfare, public 
health, library facilities), indigenous institutions (business, churches, community centers), people's percqrtions 
of these institutions, neighboriiood social organization, patterns of and reasons for mobflity, and measures of 
children's acquisition of "social capital." 
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- mount a substantial effort to "validate* the indicators that we have, as all sources of 
neighborhood dau~whether from administrative records or surveys our dau are fiill of 
inaccuracies and biases 

- engage community residents and leaders, and local and state administrative agencies in 
indicator development 

link together all of these community indicators into a national system of such indicators 

While the exercise of listing all of the possible attributes of neighborhood environment that might 
conceivably convey differences in the well-being of children is exhilarating and enlightening, they will also be 
frustrating. The lists serve as both a "counsel of perfection" and a "counsel of despair" unless they are guided 
by some organizing principles that can help us sort the inqrartant and attainable of them from the marginal and 
ephemeral. While Burton Singer's discussion gave us a start at organizing principles, we did not spend much of 
our time laying out criteria for choosing among the extensive lists. Moreover, constructing such "wish lists" is 
ultimately fiitile, unless the exercise meets head on the constraints of resources and budgets; after all, the entire 
GDP-or even the much smaller federal budget-is not available for data collection, verification, assembly, and 
research. While this is too bad, it is also true. 

Although, like the rest of you, I do researdi on children's well-being and its determinants-and, hence, 
am regularly frustrated by the lack of available or usable dau on children's attaimnents or their determinants-I 
am also an economist, and even worse, a benefit-cost analyst. I feel obliged, therefore, to bring the inevitable 
"cold shower" of resoiuce constraints and trade-offs to this discussion. I will atten^t to offer this shower in a 
way that will, hopefully, help us to make the efficient choices among the numerous options that have been set 
before us. 

To do this, let me first set out the steps necessary to subject our exercise to the rigors of a benefit-cost 
framework. They might look something like this: 

1 . Specify the objectives to be atuined. 

2. Measure the "social benefits" (maybe, even, calibrated in dollar terms) which derive from each 
objective, so that they can be traded off, one against another (or, barring this, find some way 
in which to say that some objectives are more in^rtant than others). 

3. Identify the data collection, verification or analysis proposals that could contribute to the 
objectives-establish the domam among «1iich choices will have to be made. 

4. Determine the extent to which each proposal meets technical requirements. 

5. Determine the extent to which each proposal contributes to each of these objectives (or to the 
single, monetarily denominated ovenll objective). 

6. Measure the cost of each proposal, so that the resources used from the limited budget can be 
conq>ared across the proposals. 

7. Identify those proposals that contribute the most per dollar of cost to each of the objectives, or 
to the overall monetary measure of benefits. 

In the remainder of my comments, I will discuss each of these poinu, indicating how ^plying this 
fiamework could be of help to us in deciding how best to proceed. First, however, let me note that I fiilly 
realize that performing each of these steps is clearly not feasible, and to atten^t to attain them would require as 
much of the nation's GDP as the set of indicator proposals that we have on the table. Neverthdeu, as a way of 
thinking about the problem, this framework will be helpful; hence, I persist. 
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Specify Objectives: 

Consider, first, the obiectives that participants have suggested be established to guide our choice of 
indicators on which to collect data.* Here is the list that I have found-cur indicators of children's well-being 
should be able to: 

1 . To track the nation's progress-or performance-over time in a variety of dimensions 

2. To enable comparisons among in^rtant social groups or regions 

3. To promote the public acceptance of a number of social objectives, or to influence the public's 
tlmddng 

4. To set national standards 

5. To serve as a basis for planning and targeting interventions 

6. To facilitate aad contribute to basic research on the determinants of children's attainments 

In addition, because any indicators on which we would regularly collect dau could have potentially adverse side 
effects, we should choose indicators that would not: 

7. Encourage simplistic sq;>proaches to very con^lex processes and problems 

8. Detract from realization of spatial or group concentrations of problems 

9. Encourage a punitive posture toward children (as we adults project values that we don't apply 
to ourselves onto our children) 

10. Distract from our accomplishments, and the real attainments of our children 

1 1 . Encourage measures to correct the problem, rather than prevention measures 

n y^iftmining the Social Be nefits Deriving ft?Tn Obiectives: 

Qearly all of these "outputs" of improved children's indicators are relevant to attaining a better 
society-that is, to increasing social well-being. Yet, at our conference, the goals implicit in the drive for more, 
and better, dau have not been clearly spelled out. It goes almost without mentionmg that the social value of 
increments in the attainment of any single goal-in monetary unite or any other metric-has not been thought 
through. Even more serious, we have failed to specify the relative importance among the goals, or even to 
identify the trade-offs among them. These assessmente are essential if we are to make good decisions-they are 
the second step. 

Establish the Domain of Options: 

Given that we haven't done so well in inq)lementing the first and second steps in a benefit-cost 
appraisal, how about the third step-establishmgthe domain of dau collection proposals that might contril le to 
the social objectives. In this regard, as I have indicated, we have succeeded beyond our wildest dreams~we 
have an "extravaganza" of proposals on our uble. I am sure that a very cursory count would come up with a 



n have gleaned these objectives by either lifting them directly from the papers, or by hearing them during 
our discussions. 
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couple of hundreds of them-although Greg Duncan limited himself to the FAB FIVE, Frank Furstenberg 
referred to our FAB FIVE THOUSAND. I am reminded of the "95th"-the elegant restaurant at the top of the 
John Hancock building in Chicago; only the brunch tables at the 95th would contain more delicious items than 
the catalogue we have assembled. 



Establishing Technical Criteria: 

We have also done pretty well in establishmg a list of the technical criteria by which to judge any 
proposal-*e first element in the "fourth step. Kris Moore's dozen (or is it 13) is a good start, but to them I 
would add several more that were mentioned in the papers or by participants. The full list would look like the 
following: 

Kris Moore's list 

~conq)rehensive (universal) coverage 
-positive outcomes 
-clearly defined and understandable 
-common interpretation across people 
-consistency and stability over time 
-forward looking; anticipatory 
-rigorous methods; reliable and valid 
-geogrs^hically detailed 

-susceptible to adjustment for population size and demographic composition and trends 
-cost-efficient; available at "reasonable" cost 

-reflective of inqrartant social goals; allows assessment of "social performance" 
Additional Criteria 

-objective-free of norms; not subject to ideological views of analyst 
-consistency across groups 
-capable of being reliably measured 
-available on a timely basis 

-critical in^rtance to some aspect of children's well-being 
-causally related to other indicators 

-relates to an aspect of well-being that is subject to policy intervention 

-amenable to causal analysis or disaggregation 

-exogenous with respect to other indicators 

-sufficient variability across time, groups or place 

-available for international conqMrisons 

-refleaive of con^lexity of question 

-based on accurate underlying theory 

But, while we have established numerous technical criteria, we have failed to realize that no indicator 
proposal will score well on all of them, and hence we have given ourselves but little guidance on how to 
establish some overall "technical rating" to be attached to any proposal. As a result, we are rather "at se?"^ 
when it comes to trading off gains with respect to the extent that an indicator proposal leads to attainment of o 
of the criteria, against that proposal's shortfalls in attaining another. PerbJ?)s of more concern, we have 
established but litUe agreement on the overall extent to which any particular proposal meets the technical 
requirements or criteria, and hence but littie agreement on the relative rankings of the proposals on this hmite< 
standard. 
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Measuring the Contributions of Candidat e Proposals to Objectives: 

If we were able to make progress on this 'technical requirements standard," we would be ready for the 
next step-among those proposals that pass the technical standard, how does each proposal contribute to each of 
the objectives that have been identified (or conceivably to the overall monetarily denominated social objective)? 
In fact, we have done very little by way of relating our hundreds of proposals to the objectives that we have set 
forth. This is a crucial step in maldng an informed decision; this step will have to wait until the next 
conference. 



Measuriay Costs: 

So far, all of this rdates to goals, technical requirements, and proposals. The next step is where the 
"rub" really begins-what are the costs of each of the proposals that we have on our table. Like choices on the 
tables at the 95th, each proposal exhausts some of our stomach's aq)acity~and ^>art ftom that, each may cost 
real resources-hence, each bite carries with it some opportunity cost. 

While "costing out" each proposal might seem among our easier tasks, it isn't. First, the specifics of 
each proposal must be laid out-detail, groups, areas, sanq)le size, administrative offices involved, etc.-and it 
must be specified whether the proposal involves inq)roving the accuracy of existing statistics or developing new 
information. Then, the costs of implementing the proposal can at least be thoughtfully considered. But even 
then, the nature of "costs" can be con^lex, and difficult to understand. While the dollar costs of inq>lementing 
a new longitudinal survey might be estimated, the iiiq)licit costs of adding to (or modifying) an existing survey 
are also relevant. For example, as all survey researchen with whom I am familiar will be quick to remind us, 
adding questions or modules to any existing survey in^>lies displacing some questions or modules that are now a 
part of the survey. When gathering new data means sacrificing existing series, real opportunity costs arc 
involved, and they are likely to be more difficult to measure or even discern than the dollar costs of the 
resources necessary to gather new data. 



Choosing the Most Cost-Beneficial Proposals: 

Only if we have followed these steps will be able to reliably undertake the final step of the required 
analysis-choosing those indicator proposals that both fit into our limited budget and nuudmize the contributions 
to the objectives for this enterprise that we have stipulated. And, it must be enq)hasized, that is the ultimate 
purpose of our meeting together. We have left ourselves a very long way from this "holy grail." 

While I hate to be the represenutive of the "dismal science" to carry this bad news, I would enq)hasize 
that its basic message is essential. While we might feel good by observing the set table at the 95th, we bad 
better know well our capacities, our budgets, our tastes, and the extern to which each of these delicacies absorb 
our capacity and our budget, and the way that eadi of them interact with the others. So far, we have only seen 
the "smorgasbord" set; next comes the difficult job of evaluation, and decision. Those tashi can be the subject 
of a third conference. 

In concluding, I would like to leave you with three thoughts. 

First, I would offer a suggestion to the conference organizers and volume editore. Implicit in the 
several papers' at the conference are two quite different perspectives. One .-^rspective is aimed at identifying 
"thermometer-type" indicators that could serve as the basis for regular reports of trends in children's well-being. 
The other perspective sought to identify the date and variables necessary for researchers to \mpio\e their 
analysis of the determinanu of children's success, and the process by which quality children are produced. 
These two approaches are quite distinct in terms of their purposes, the standards that are ^lied, and their 
cosu. In my view, a first step would be to distinguish-and segrcgate-these quite different proposals. 
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Second, I would like to suggest a *FAB ONE*~a proposal for children's indicators that could be taken 
immediately and at low cost. Unless I am quite wrong, I believe that within 6 weeks after every March Current 
Population Survey we could report a wide variety of children's indicators that would be: 

-annual 
-reliable 
-highly visible 
-uniform 

-capable of disaggregation 

These indicators would include: 

-percentage of children living in poor families 
-percentage of children living mih one parent 
-percentage of children living with an unen^loyed parent 
-percenuge of children living with a jobless and ill parent 
-percentage of children living in a family on welfare 

In my view, the regular publication of such single indicators would-like the nation's unen^loyment rate or the 
prime rate or the poverty rate-serve a valuable first step in assembling regular children's indicator reports. 

One fmal point derives from reflecting on those institutions that no longer seem to work in American 
society, and the difRculty of government in replacing them. The task of replacing families, churches and 
neighborhoods by government is an enormous one. I am struck by how difficult and expensive it is to bring a 
young person, a child, to a position, in which *work will work.* I believe that an effort to accurately measure 
the full cost of such e^orts would be instructive and valuable. In a very real sense an estimate of this cost is a 
children's indicator. My sense is that our own personal experience gives some insight into the magnitude of 
these costs. How do we do it for our kids? Well, first we give them lots of education with monitoring and 
advice and expectations and parental participation in schools. Then, y^ea they finish their schooling, we 
support them for a time while they get their heads together. Sometimes they engage in job search, sometimes 
they ski, sometimes they travel. Following this, we actively and in a bne-on-one relationship, help them with 
job search. We help them prepare a resume, we put them into touch with friends and acquamtances, we help 
them to prepare for job interviews-all so they can find their own special niche in the world of work. Finally, 
we often support them in moving to another location, often far from our home if that is where the best 
opportunity for them is. All of these things are enormously expensive in terms of both time and other forgone 
opportunities. It would do well for this society to have to confront head-on the magnitude of these costs. 

The main lesson, I fear, is that doing this e^ectively is costly, very costly. There is no way to do it on 
the cheap. The realization of this truth makes more distressing our talk of making welfare recipients self- 
sufficient with a reform that will not violate budget neutrality. 
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Rapporteur's Comments for the 
Conference on Indicators of ChUdren's Wdl-Being 
Sheldon Danziger, University of Michigan 



In my view, its p^rs at this conference raised more questions than they answered. As a result, my 
remarks will take the form of a set of questions. 

1. What Is The Question? Wha t Onals Should rnme Out of This Conference? 

Is the purpose to gather as many indicators as possible? Or is it to identify a iHMl) nvni^>fr 9f KtY 
indicators which are the best predictors of those childtens* outcomes that we care most about? If it is the latter, 
should we target date-gathering resources on fewer indicators and make certain that we have enough geographic 
deteil; enough race/ethnic dettil, etc.? Should we worry more about getting indicators of the aaSSS of 
problems of children or the indicators of the cnnseouences of these problems? 

Do we need additional research to answer these questions and tell us what indicators we need? Or, are 
we now prevented fitom doing importont research by not having enough indicators? Frank Furstenberg has 
rejected the view that our problem is simply one of insufficient date. He teUs us that we need to devote much 
more attention to trying to understend the processes that determme chUd outcomes. That is, before we go out 
and gather a lot more indicators at the census tract level, we should decide whether we really think that tracts 
proxy neighboihoods and thus are a relevant universe of analysis. 

I take as my rapporteur's task the following charge~if I had some money to invest in studying 
chUdren's well-being, where would I spend it, based on what I have heard at the conference over the last two 
days and on what the authors have written in their ps^rs? 

2. Whv Should We rate about Indicators of Child Well-Bcing? 

What is unique about child indicators? Are there any indicators for children that truly differ from those 
at other points in the life-cycle? Or should the indicator be the same, but we should care more about a problem 
for children because they are dependent and because the effects of the problem can be long-lasting? For 
example, is there a difference in measuring child poverty vs. poverty among the elderly or in measuring 
domestic violence vs. child abuse? 



The Pritnacv of Poverty 

Should child poverty be the key economic indicator? Is it a good enough proxy for other dimensions 
childrens* well-being? What about indicators such as percentage of children who are not covered by health 
insurance, or the percentoge of chUdren who live with unemployed or jobless parents? Do we already have 
enough date? Can we get most of what we want by making relatively minor changes in the indicators we 
currently gather? I'm not convinced that the pa^rs, taken as a whole, gave me the answers I seek. Each 
author seems to have his/her favorite questions and favorite date gathering needs. 



3. Pfu/areoftheMean 

In a world of rising inequality both within groups and between groups, indicators of central tendency 
become less helpful since the last two decades have been ones of rising inequality, this suggests that we need to 
pay more anention to the distributions of our indicators. For example, we need to know both the mean and the 
variance of test scores. On some indicators, there has been no change on average in the past two decades m the 
well-being of children. But while the mean or median has been constant, the distribution has changed 
dramatically-some children have done very well and others have done very badly. 
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4. Beware of Indicators That Do N<?t Indicate What Many Think Thev Indicate 

Some of the p^rs teach us that too much data can be bad for our understanding, especially if those 
dau are not drawn from random saasples of children. This is particularly a problem with measures based on 
administrative records. For exanq)le, we do not know if child well-being is increased or decreased if we get an 
increase in the welfare rolls or the foster care rolls and all we have are administrative records. In this case, we 
cannot distinguish between two opposite situations~(a) The caseloads may have grown solely because people 
who previously had a problem were not being aided at all by the system. In this case, growth indicates that the 
system has become more effective in serving its intended caseload, (b) The case loading may have grown 
because the number of people with the problem that the service addresses has increased. In this case, the 
problem has worsened but the service delivery system has remained fixed. 

Gary Sandefur goes further and warns us that some popular indicators are not indicators at all. For 
example, the public considers trends in the percentage of children bom out-of-wedlock and the percentage living 
in mother-only families as indicators of the breakdown of the fanuly. Some, such as rharles Murray, might go 
even further and argue that these childbearing and family trends represent a threat to civilization as we know it. 
Sandefur argues that these are not direct indicators of child well-being. Sandefur offers healthy/unhealthy child 
as a direct indicator and others offer educational attainment because these measures directly distinguish between 
individual children. Sandefur's definition seems to imply that child poverty (my favorite indicator) is also not a 
direct indicator because it is not necessarily the case that a child in a poor family is doing worse than a child in 
a nonpoor family. I would disagree with Sandefur, however. I would point the recent book by Sara 
McLanahan and Sandefur, which says that in addition to poverty, family structure is one of the most inqmrtant 
predictions of a child's outcomes as a young adult. Even if we label both child poverty and family structure as 
indirect measures, they are more important, in my view, than most of the other indicators we have discussed. 

S. Is the Problen i ? nf Indicators or a Lack of Con cern about the Indicators? 

The Miringoffs lament that the government does not release information on social indicators on as 
timely a basis as it does for economic indicators. Nonetheless, after two days at this conference I am 
overwhelmed by the amount of information we do have. We have, for exanq)le, a very large array of child 
health indicators. I do not think it would change public opinion or public policy if these iixlicators were made 
available on a quarterly or monthly basis. The data we have demonstrates that child health in Amenca is not as 
good as it should be. This has not caused us as a nation to move quickly to inq>rove child health in the way 
that the Federal Reserve has just moved quickly to raise interest rates to reduce the fear of inflation in the bond 
market. 

In this regard, I interpret the situation NOT as the Miringoffs do. They suggest that we ^blpk 
differently about the well-being of children than we do about the well-being of the economy because we do not 
have timely adequate dau. I argue the reverse. That is, because we have fimdamentally different concerns 
about the well-being of children than we do about the well-being of the economy, we collect daU differently. If 
Americans were to suddenly have a change of heart, and think like the Swedes, we would find that we have 
plenty of data on which to base changes in child health policies. 

Hugh Heclo, in his essay 'Poverty Politics," in Confronting P overty: Prescriptions for Change, asks 
why antipoverty policy is so problematic. His lessons in the overall poverty context arc relevant for those of us 
at this conference who view a renewed interest in child indicators as a wiy to further antipoverty policies in a 
variety of health, education, and income support areas. As I read his quote, you should insert "children" where 
he says "the poor." 

At least three basic facts shape everything else that might be said about the politics of poverty 
policy. The first is that poor people do not have much power. Second, America's poverty 
agenda is now inseparable from its racial debate. Finally, antipoverty efforu are bound up 
with the sute of the economy. . . to economic good times it is politically easier to feel 
generous. . . But the poor can alto appear more aberrant, and fewer people may identify with 
their situation, to economic hard times more people may feel their economic situation 
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precarious ... But antipoverty efforts also become harder to pay for . . . Economic 
conditions offer one simple shortcut through the politics of poverty (pp. 397-398). 

The recent elections arc consistent with Heclo's conclusion and suggest to mc why we arc not likely to 
get more antipoverty policies or more children's policies no matter how many indicators of child well-being that 
we gather. The past two decades have been marked by slow economic growth and rumg poverty and 
inequality. Married-couple famUies have managed to do better than others because they had the abUity to send 
an extra earner into the labor force or to have an earner woric longer hours. But married-couple families on 
average have not fared well, certamly not relative to what they expected to have, and certainly not relative to 
the kind of income growth their parents experienced at similar points in the life cycle. There is no longer much 
more room for additional work in married-couple families. So these families seem to have decided, "Give us a 
middle-class tax break and pay for it by cutting spending on social programs." They probably have not realized 
that many of the program cuts wiU affect programs that benefit them-such as subsidies for higher education, 
niey do not seem to be troubled by the prospect that the spending cuts will hurt other people's children, 
especially children in mother-only fanulies and nunority children. TTiey do not take the long view and conclude 
that structural problems related to a changing economy arc at the core of their own economic problrais. Ramer 
they blame the government for their economic problems and view social welfare programs not as offsettmg toe 
hardship generated for miUions in the labor maricet but as using their tax money to support nonworking welfare 
recipients. They go on to blame government and the behavior of other parents for most of the social problems 
ibat concern them-Ksrimc, out-of-wedlock childbearing. poor health of chUdren, etc. Heclo's conclusion is not 
that we have a paucity of indicators of chUd well-being, but that we have a paucity of social concern. 
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